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Abstract 


Several  studies  have  shown  that  pigeons  prefer  variable 
over  fixed  schedules  of  rei nforcement .  The  present  research 
examined  preference  for  variable  versus  fixed  schedules  of 
secondary  reinforcement  in  humans.  Experiment  1  used  a 
concurrent -chai ns  procedure  allowing  subjects  to  choose 
between  mutually  exclusive  f i xed- i nterva 1  (FI)  30-sec  and 
mi xed- i nterva 1  (MI)  30-sec  alternatives.  The  effect  of 
randomness  in  the  MI  schedule  was  also  examined;  the 
component  intervals  were  presented  in  either  the  standard 
random  order  or  in  a  nonrandom  alternating  order  (an  "AMI" 
30-sec  schedule).  Experiment  2  used  a  similar  procedure  to 
directly  compare  preference  for  MI  30-sec  versus  AMI  30-sec 
alternatives.  The  results  from  these  two  experiments  were 
inconsistent  and  contr adi ctory .  Experiment  3  used  an 
entirely  different  procedure  and  presented  subjects  a  single 
choice  between  fixed  and  variable  alternatives  with  much 
larger  time  delays  and  reinforcement  amounts,  i.e.,  subjects 
could  receive  ten  dollars  at  the  end  of  one  month  (the  fixed 
alternative),  or  they  could  gamble  on  receiving  it 
immediately  or  in  two  months  (the  variable  alternative). 
Nine  out  of  ten  subjects  chose  the  variable  alternative. 
These  results  are  related  to  the  ability  of  human  subjects 
to  respond  in  terms  of  large  reinforcers  presented  at  long 
delays . 
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Intnoduct ion 


Choice  and  preference  are  fundamental  aspects  of  human 
behavior.  Individuals  characteristically  distribute 

responses  among  many  alternative  sources  of  rei nforcement . 

✓  ' 

Alternatives  may  differ  in  the  type  and  amount  of 
rei nforcement ,  as  well  as  in  the  contingencies  of 
rei nforcement . 

Preference  for  different  types  of  contingencies  may  be 
an  important  factor  in  human  behavior.  For  example,  Emerson 
(1972)  has  proposed  that  humans  prefer  variable  over  fixed 
schedules  of  social  rei nforcement .  Individuals  who  deliver 
social  reinforcement  on  more  variable  schedules  may  thereby 
exert  greater  influence  over  the  behavior  of  others.  Thus, 
schedule  preference  may  be  a  determinant  in  many  social 
phenomena  such  as  dominance,  leadership,  and  charisma.  This 
suggests  that  an  individual's  distribution  of  behavior  may 
be  altered  by  changing  the  operating  contingencies  in  a 
choice  situation.  In  summary,  an  analysis  of  schedule 
preferences  in  humans  may  have  important  implications  for 
the  prediction  and  control  of  human  behavior.  The  present 
research  investigated  human  preference  for  variable  versus 
fixed  schedules  of  reinforcement. 

The  most  common  paradigm  used  to  measure  schedule 
preference  is  the  two- link  concurrent -chai ns  procedure 
(Autor,  1969).  In  the  initial  link,  the  subject  responds  on 
two  concurrently  available  alternatives,  usually  programmed 
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on  equal  var i ab 1 e- i nterva 1  (VI)  schedules.  Meeting  the 
requirement  on  an  initial  link  alternative  results  in  the 
presentation  of  a  terminal  link  schedule  of  reinforcement. 
This  is  signalled  by  an  exteroceptive  stimulus  change,  while 
the  other  alternative  becomes  inoperative.  Thus,  in  the 
initial  link,  the  subject  chooses  between  two  mutually 
exclusive  terminal  link  schedules.  Preference  between 
terminal  link  schedules  is  measured  by  relative  rate  of 
responding  on  the  initial  link  schedules. 

A  second  paradigm  for  measuring  schedule  preferences 
utilizes  the  switching  procedure  devised  by  Findley  (1958). 
Responses  on  a  change-over  (CO)  alternative  change  the 
schedule  of  rei nforcement  in  effect  on  a  second  alternative. 
For  example,  in  the  procedure  employed  by  Sherman  and  Thomas 
(1968),  if  the  first  response  was  on  the  CO  alternative,  the 
subject  could  not  switch  back  to  the  original  schedule. 
They  were  then  committed  to  receiving  rei nforcement  on  the 
new  schedule.  Similarly,  if  the  first  response  was  on  the 
presented  schedule,  the  subject  was  committed  to  receiving 
rei nforcement  on  that  schedule.  Thus,  the  first  response 
was  a  choice  response,  and  was  similar  to  the  initial  link 
choice  phase  in  the  concurrent -chains  procedure. 

Both  of  these  paradigms  separate  responding  for  a 
schedule  from  responding  on  a  schedule.  Response  rates  on  a 
schedule  are  not  good  measures  of  preference  for  alternative 
contingencies  of  reinforcement  since  they  may  be  heavily 
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influenced  by  characteristics  of  the  schedule  itself.  The 
concur  rent -chai ns  procedure  has  an  additional  advantage 
since  it  provides  a  measure  of  response  rate  on  the  initial 
links  (in  addition  to  CO  responses).  The  Findley  procedure 
may  be  considered  similar  to  a  chains  procedure  with  FR 
initial  link  schedules. 

The  concurrent-chains  procedure,  with  first  link  VI 
schedules,  can  be  regarded  as  an  extended  version  of  a 
simple  concurrent  VI  schedule.  A  well-known  finding  in  the 
experimental  analysis  of  choice  behavior  is  that  pigeons 
match  relative  rate  of  responding  on  two  concurrently 
presented  VI  schedules  to  relative  rate  of  rei nforcement 
( Fler  rnstei  n ,  1961).  This  effect,  known  as  the  matching  law, 
has  been  replicated  in  a  variety  of  situations  and  species 
(de  Villiers,  1977),  including  humans  (Pierce  &  Epling, 
1983).  Using  the  concur rent -cha i ns  procedure,  Herrnstein 
(1964a)  found  that  pigeons  matched  relative  rate  of  initial 
link  responding  to  relative  rate  of  terminal  link 
rei nforcement ,  when  both  terminal  links  consisted  of  VI  and 
VR  schedules. 

In  another  experiment,  Herrnstein  (1964b) found  that 
when  one  terminal  link  was  a  f i xed- i nterva 1  (FI)  schedule, 
pigeons  responded  for  a  VI  alternative  more  than  predicted 
by  relative  rates  of  rei nforcement .  More  specifically, 
although  the  arithmetic  means  of  each  schedule  were  equal, 
e.g.,  VI  30-sec  and  FI  30-sec,  pigeons  strongly  preferred 
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the  VI  alternative.  Subsequent  research  found  that  animals 
preferred  variable  over  fixed  schedules,  whether  the 
schedules  were  interval  (Davison,  1969,  1972;  Hursh  & 
Fantino,  1973;  Killeen,  1968;  NavaricK  <5  Fantino,  1972, 
1975),  ratio  (Fantino,  1967;  Hendry,  1969;  NavaricK  <$ 
Fantino,  1972,  1975;  Rider,  1983b;  Sherman  <§  Thomas,  1968), 
or  response  independent  delay  of  reinforcement  (Cicerone, 
1976;  Rider,  1983a).  With  the  exception  of  Sherman  and 
Thomas  who  employed  the  Findley  switching  paradigm,  these 
studies  used  the  concur rent -cha i ns  procedure.  Finally,  in 
most  of  these  experiments  the  variable  alternative  consisted 
of  a  "mixed"  schedule:  a  type  of  variable  schedule  which 
controls  for  the  number  and  size  of  component  values  in  the 
schedule.  For  example,  a  mi xed- i nterva 1  (MI)  30-sec 
schedule  might  consist  of  an  equal  number  of  15-  and  45-sec 
intervals  presented  in  random  order . 

Herrnstein  (1964b)  accounted  for  preference  for  the 
variable  schedule  by  suggesting  that  a  temporal  scaling 
factor  may  be  involved.  In  this  account,  the  shorter 
component  intervals  in  the  VI  schedule  exert  a 
disproportionate  influence  on  responding,  i.e.,  they  are 
"weighted"  more  heavily.  A  number  of  studies  have  attempted 
to  derive  a  transformat  ion  rule  on  this  basis  (Davison, 
1969,  1972;  Hursh  &  Fantino,  1972;  Killeen,  1968),  but 
results  have  been  inconsistent.  For  example,  Killeen 
suggests  a  harmonic  transformat  ion  of  the  intervals,  while 
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Davison's  (1969)  results  are  best  described  by  a 
transformat i on  of  the  reciprocals  of  the  intervals  to  the 
third  power.  Frankel  and  Vom  Saal  (1976)  conclude  that  "no 
consistent  transformat  ion  has  been  found  which  is  adequate 
to  account  for  more  than  a  limited  set  of  data"  (p.  71). 
Nevertheless,  all  suggested  transformations  weight  shorter 
intervals  more  heavily,  and  the  importance  of  this  factor 
seems  well-established. 

In  addition  to  the  temporal  scaling  factor,  Herrnstein 
(1964b)  suggests  his  results  are  similar  to  gambling 
behavior  in  humans:  "the  gambler  may  be  like  the  pigeons  in 
the  present  experiment;  they  were  'placing  their  bets' 
consistently,  if  not  wisely"  (p.  181).  In  other  words, 
preference  for  VI  over  FI  results  from  the  possibility  of 
"winning"  a  shorter  interval  to  reinforcement  on  the  VI 
alternative.  Thus,  preference  for  VI  may  be  largely  a 
function  of  the  random  presentation  of  shorter  intervals  on 
VI,  such  that  there  is  some  probability  of  their  occurrence 
whenever  the  VI  schedule  is  selected.  There  have,  however, 
been  no  studies  that  have  directly  manipulated  this  factor. 

With  respect  to  human  preference  for  variable  versus 
fixed  schedules,  research  has  been  limited  and  inconclusive. 
Repp  and  Dei tz  (1975)  used  a  switching  procedure  to  examine 
preference  for  VR60  versus  FR60  schedules  of  monetary 
reinforcement.  The  VR  and  FR  schedules  alternated 
automatically  after  each  rei nforcement ,  and  a  CO  response 
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was  necessary  to  reinstate  the  previous  schedule.  All  four 
subjects,  age  10  to  12,  switched  to  the  VR  schedule  more 
often  than  to  the  FR  schedule.  Weiner  (1966)  used  a  similar 
procedure  to  examine  preference  in  four  adults  for  VR40 
versus  FR40  schedules  of  point  delivery.  However,  Weiner 
found  no  preference  for  either  schedule;  the  subjects  rarely 
made  a  CO  response,  and  simply  responded  on  the  two 
schedules  as  they  were  automatically  presented  in 
alternating  order .  In  addition  to  using  older  subjects,  the 
Weiner  study  differed  from  Repp  and  Deitz  in  other  ways. 
Most  notably,  subjects  were  instructed  to  "get  the  highest 
score  possible".  Relative  to  this  strategy,  responding  on 
the  CO  alternative  would  waste  time  and  reduce  total  points 
earned . 

There  have  been  no  human  studies  using  the 
concurrent -chai ns  procedure.  Significantly,  this  is  the 
most  commonly  used  procedure  in  animal  research  where 
results  have  been  fairly  consistent.  Furthermore,  as 
previously  noted,  human  responding  on  concurrent  VI 
schedules  has  been  shown  to  be  similar  to  that  found  in 
animals  (Pierce  &  Epling,  1983).  Given  that  the 
concur  rent -chai ns  procedure  is  an  extended  version  of  a 
simple  concurrent,  it  may  be  that  schedule  preferences  in 
humans  would  more  closely  match  the  results  obtained  with 
animals  if  this  procedure  was  used.  The  present  research 
examined  preference  in  adult  humans  for  variable  versus 
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fixed  schedules  of  monetary  reinforcement  using  a 
concur  rent -chai ns  procedure.  It  was  predicted  that  relative 
rate  of  responding  in  the  initial  links  would  be  greater  for 
the  alternative  leading  to  the  variable  schedule.  The  role 
of  "gambling"  in  determining  variable  schedule  preference 
was  also  examined  by  manipulating  the  randomness  of  the 
variable  schedule.  It  was  predicted  that  the  variable 
schedule  would  be  more  preferred  when  the  schedule 
components  were  presented  in  random,  rather  than  nonrandom, 
order . 


Experiment  1 

Experiment  1  used  a  concur  rent -cha i ns  procedure  to 
examine  preference  in  humans  for  variable  versus  fixed 
schedules  of  monetary  reinforcement.  The  variable 
alternative  consisted  of  an  MI  30-sec  schedule  with 
component  intervals  of  5  and  55  seconds.  The  fixed 
alternative  was  an  FI  30-sec  schedule.  It  was  hypothesized 
that  relative  rate  of  responding  would  be  greater  on  the 
initial  link  alternative  leading  to  the  MI  schedule. 
Experiment  1  also  examined  the  "gambling"  effect  in 
determining  preference  for  the  variable  schedule.  Across 
conditions,  two  types  of  MI  schedules  were  employed:  a 
standard  MI  schedule  with  5  and  55  second  components 
presented  in  random  order,  and  an  "alternating 
mi xed- interval"  (AMI)  schedule  with  the  two  components 
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presented  in  alternating  order.  If  the  "gambling"  effect  is 
important  then  subjects  should  exhibit  greater  preference 
for  MI  over  FI  than  for  AMI  over  FI.  This  result  would 
indicate  that  the  random  presentation  of  component  intervals 
is  a  determinant  of  human  preference  for  variable  schedules. 

Method 

Subjects 

Four  experimentally  naive  female  university  students 
served  as  subjects.  They  had  answered  a  poster  announcing 
payment  for  research  participation,  and  were  selected  mainly 
on  the  basis  of  an  expressed  need  for  money.  Each  subject 
signed  a  contract  stating  that  remuneration  would  be  based 
on  points  earned  during  the  experiment,  plus  a  ten  dollar 
bonus  for  completion  of  all  scheduled  sessions. 

Apparatus  and  Setting 

Subjects  responded  on  the  panel  displayed  in  Figure  1. 
Response  buttons,  which  could  be  illuminated,  were  1.5  cm 
square  and  required  a  force  of  500  grams.  The  initial  link 
buttons  were  both  white,  while  left  and  right  terminal  link 
buttons  were  blue  and  yellow,  respectively.  Consumatory 
response  buttons  were  colored  to  match  the  terminal  link 
buttons,  and  were  included  to  increase  sensitivity  to 
contingencies  (Matthews,  Shimoff,  Catania,  &  Sagvolden, 
1977).  Left  and  right  counters  registered  total  points 
earned  on  respective  terminal  link  schedules.  For  each  pair 
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of  buttons/counters,  the  left-side  alternative  was  labelled 
"A",  and  the  right-side  alternative  was  labelled  "B" . 

Subjects  sat  facing  the  response  panel  that  was 

positioned  on  top  of  a  desk.  Since  pilot  subjects 

complained  of  boredom,  a  radio  provided  music  during 
experimental  sessions.  Colbourne  programming  equipment 
located  in  a  separate  room  controlled  presentation  of  events 
and  recorded  subjects'  responses. 

Procedure 

Prior  to  the  first  session,  subjects  were  asked  to  read 
the  following  list  of  instructions: 

"1.  Do  not  press  any  buttons  until  you  hear  the  beep  which 
signals  the  start  of  the  experiment. 

2.  In  this  study  you  earn  points  by  pressing  the  various 
buttons.  Press  the  buttons  only  when  they  are  lit. 
When  2  buttons  are  lit  simultaneously,  you  can  press 
whichever  one  of  them  you  choose. 

3.  Each  time  you  earn  a  point,  it  is  recorded  on  one  of 
the  counters  on  the  panel.  Each  point  you  earn  is 
worth  8  cents  and  your  task  is  to  earn  as 

much  as  possible. 

4.  Both  the  start  and  the  end  of  the  session  are 
indicated  by  the  sound  of  the  beeper.  When  the 
beeper  sounds  to  end  the  session,  please  wait  in 
your  chair  until  the  researcher  arrives. 


.  . 
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5.  If  you  have  any  questions  about  the  procedure,  please 
reread  these  instructions." 

Concurrent -chai n  schedules,  outlined  in  Figure  2,  were 
the  operative  contingencies.  During  the  choice  phase,  two 
white  initial  link  buttons  were  illuminated  and  independent 
VI  schedules  were  operative  on  each  button.  A 
change-over -de 1  ay  (COD)  was  also  in  effect  during  this 
phase.  The  COD  stipulated  that  a  delay  of  n  seconds  would 
occur  following  a  change-over  from  one  button  to  the  other. 
Responses  during  the  delay  were  not  effective  in  meeting  the 
VI  schedule  requirements.  When  the  subject  completed  the 
requirement  on  either  initial  link,  both  buttons  turned  dark 
and  one  of  the  terminal  links  became  operative.  Meeting  the 
VI  requirement  on  the  left  initial  link  button  resulted  in 
the  illumination  of  the  left  (blue)  terminal  link  button, 
while  meeting  the  VI  requirement  on  the  right  initial  link 
button  resulted  in  the  illumination  of  the  right  (yellow) 
terminal  link  button.  Completing  the  schedule  requirement 
on  either  terminal  link  resulted  in  that  button  going  dark 
and  the  illumination  of  its  associated  consumatory  response 
button.  A  single  response  on  this  button  then  turned  it 
dark  and  registered  a  point  on  the  appropriate  counter.  The 
two  initial  link  buttons  were  then  illuminated  to  initiate 
another  choice  phase. 

The  initial  links  consisted  of  VI  10-sec  schedules. 
While  research  with  pigeons  has  typically  used  VI  60-sec 


schedules  (e.g.,  Davison,  1969),  pilot  subjects  for  the 
present  study  were  consistently  insensitive  to  terminal  link 
schedules  when  initial  links  were  this  long.  Terminal  link 
alternatives  constituted  the  independent  variable;  the 
left-side  terminal  link  was  always  an  FI  schedule,  while  the 
right-side  terminal  link  consisted  of  either  an  FI,  MI,  or 
AMI  schedule.  The  AMI  30-sec  schedule  consisted  of  5  and  55 
second  intervals  presented  in  alternating  order .  The  MI 
30-sec  schedule  consisted  of  an  equal  number  of  5  and  55 
second  intervals  presented  in  random  order ,  with  the 
restriction  that  there  could  not  be  more  than  four 
consecutive  presentations  of  the  same  interval  value. 
Dependent  measures  consisted  of  relative  rates  of  responding 
on  initial  link  schedules,  absolute  rates  of  responding  on 
initial  and  terminal  link  schedules,  and  total  and  relative 
number  of  entries  into  each  terminal  link. 

The  sequence  of  experimental  conditions  for  each 
subject  is  presented  in  Table  1.  A  modified  ABCB  reversal 
design  was  used,  counterbalanced  across  subjects.  Subjects 
A3  and  A4  also  received  Condition  D  comparing  preference  for 
FI  30-sec  versus  FI  15-sec  schedules.  This  was  because, 
during  early  sessions,  these  subjects  developed  a 
stereotyped  response  pattern  of  switching  back  and  forth 
between  alternatives.  Condition  D  was  initiated  to 
determine  if  these  subjects  were  at  least  sensitive  to 
different  terminal  link  pay-offs,  as  would  be  indicated  by 
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preference  for  the  richer  schedule.  The  stereotyped 
switching  pattern  was  also  the  reason  for  the  longer  COD 
used  with  Subject  A4;  it  was  increased  first  to  four  seconds 
to  try  and  prevent  the  occurrence  of  the  stereotyped  pattern 
that  Subject  A3  had  previously  demonstrated.  When  the 
pattern  developed  anyway,  the  COD  was  increased  to  15 
seconds  in  session  12  to  try  and  eliminate  it.  With  a  15 
second  COD,  switching  back  and  forth  would  reduce  overall 
earnings,  and  continued  switching  would  strongly  indicate 
that  choice  was  not  under  control  of  monetary  reinforcement. 

Subjects  were  given  three  sessions  per  day.  Sessions 
lasted  50  minutes  with  10  minute  breaks  between  sessions. 
All  subjects  initially  contracted  to  participate  for  four 
consecutive  days,  and  Subject  A3  agreed  to  return  for  a 
fifth  day.  At  the  end  of  the  final  session  for  the  day, 
total  points  registered  were  recorded,  and  the  subject  was 
informed  of  monetary  earnings.  On  average,  subjects  earned 
$6.00  per  session. 


Results  and  Discussion 

Results 

Results  for  each  session  are  based  on  data  from  the 
entire  session.  Preference  for  terminal  link  schedules  was 
determined  by  relative  rate  of  response  on  the  initial  link 
alternatives.  The  number  of  responses  on  one  alternative 
was  divided  by  the  total  number  of  responses  on  both 
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alternatives.  Similarly,  relative  entries  into  terminal 
links  consists  of  number  of  entries  into  one  terminal  link 
divided  by  total  number  of  entries  into  both  terminal  links. 

Relative  rates  of  response  in  the  initial  link  are 
displayed  in  Figure  3.  The  first  three  sessions  constituted 
a  baseline  condition  with  FI  30-sec  schedules  presented  in 
both  terminal  links.  During  this  condition,  relative  rates 
were  close  to  .5  for  all  subjects.  This  means  that  subjects 
were  responding  equally  on  both  alternatives,  suggesting  an 
absence  of  position  bias. 

Two  subjects  demonstrated  schedule  preferences. 
Subject  A2  strongly  preferred  MI  over  FI,  with  exclusive 
responding  for  the  MI  alternative  during  sessions  8  and  9. 
On  the  other  hand,  she  showed  no  preference  for  AMI  over  FI. 
In  fact,  during  session  6,  relative  response  rate  for  the 
initial  link  alternative  leading  to  AMI  was  .33,  indicating 
greater  responding  for  the  FI  alternative.  Subject  A1  also 
preferred  MI  over  FI,  and  not  AMI  over  FI.  However,  the 
effect  was  weak.  While  preference  for  MI  increased  steadily 
during  the  first  presentation  of  Condition  B  (sessions  4  to 
6),  it  was  less  consistent  during  the  second  presentation  of 
that  condition  (sessions  10  to  12).'  Relative  responding  for 
MI  decreased  from  .56  in  session  10  to  .49  in  session  11, 
before  finally  recovering  to  .63  in  session  12.  It  should 
be  noted  that,  with  both  subjects,  conditions  were  sometimes 
changed  before  preferences  had  stabilized.  Unfortunately, 
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s  tabi 1 i ty 

could  not  be  used 

as  a 

cr i ter  ion 

for 

changing 

condi t i ons 

because  of  the 

limited  number 

of 

sessions 

subjects 

participated  in. 

Thi  s 

was  also 

the 

case  in 

Experiment  2. 

Relative  response  rates  for  Subjects  A3  and  A4  indicate 
no  consistent  schedule  preferences  in  any  of  the  conditions 
presented  to  them.  Rates  deviated  little  from  .5  in  all 
sessions,  especially  for  Subject  A4.  Informal  observation 
revealed  that  both  subjects  had  a  consistent  pattern  of 
switching  back  and  forth  between  alternatives;  they  would 
respond  on  the  left  side  alternatives,  earn  a  point,  then 
respond  on  the  right  side  alternatives,  earn  a  point,  then 
back  to  the  left  side,  etc.  This  pattern  was  maintained 
even  with  FI  15-sec  versus  FI  30-sec  alternatives  during 
final  sessions,  and,  for  Subject  A4,  when  the  COD  was 
increased  to  15  seconds  in  session  12. 

Additional  results  are  presented  in  Table  2.  For  most 
subjects,  relative  entries  into  terminal  links  parallel 
relative  rates  of  responding  in  the  initial  links.  That  is, 
a  higher  proportion  of  responses  to  an  initial  link 
alternative  usually  resulted  in  more  frequent  selection  of 
the  associated  terminal  link  schedule.  The  significant  drop 
in  total  number  of  terminal  link  entries  by  Subject  A4  in 
session  12  is  the  result  of  the  15  second  COD  operative 
during  this  session.  The  subject  continued  to  switch  back 
and  forth  between  alternatives  despite  the  long  COD,  and 
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thus  spent  more  time  in  the  initial  links. 

With  respect  to  absolute  response  rates,  terminal  link 
rates  by  Subject  A1  were  consistently  higher  on  MI  and  AMI 
than  on  FI,  and  usually  higher  on  MI  than  on  AMI.  However, 
absolute  rates  by  Subjects  A2 ,  A3,  and  A4  did  not  differ 
between  terminal  link  schedules.  There  were  also  between 
subject  differences  in  absolute  response  rates;  rates  were 
very  low  for  Subject  A2 ,  especially  on  terminal  link 
schedules,  and  extremely  high  for  Subject  A4.  Informal 
observation  indicated  that  Subject  A4  generally  responded  at 
a  high  rate  throughout  the  rei nforcement  interval,  while 
Subject  A2  usually  waited  a  period  of  time  before 
responding.  Interestingly,  Lowe  (1979)  notes  that  human  FI 
performance  is  characterized  by  two  main  patterns:  a  low 
rate  pattern  consisting  of  a  few  responses  emitted  at  the 
end  of  the  interval,  similar  to  Subject  A2' s  behavior,  and  a 
high  rate  pattern  consisting  of  a  high  undifferentiated  rate 
throughout  the  interval,  similar  to  Subject  A4' s  behavior. 

Discussion 

Only  two  subjects  behaved  in  accordance  with  the 
predictions.  Subject  A2  developed  strong  preference  for  MI 
over  FI,  and  showed  no  preference  for  AMI  over  FI.  Subject 
A1  showed  a  similar  pattern,  though  preference  for  MI  was 
inconsistent  during  the  second  presentation  of  the  MI  versus 
FI  condition.  It  is  difficult  to  account  for  this 
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inconsistency,  though  it  may  due  to  an  interaction  with  the 
preceding  AMI  versus  FI  condition.  The  results  in  general 
suggest  that  preference  for  variable  over  fixed  schedules 
may  not  be  as  prevalent  a  phenomenon  in  humans  as  i t  is  in 
animals.  When  such  preference  occurs,  however,  the  random 
presentation  of  component  intervals  does  seem  to  be  a 
determining  factor. 

Subjects  A3  and  A4  showed  no  preference  for  any  of  the 
schedule  alternatives  presented  to  them.  This  occurred 
despite  the  use  of  relatively  short  initial  link  schedules, 
which  should  have  enhanced  differential  preference  for 
terminal  link  alternatives  (Hursh  &  Fantino,  1974;  Frankel  & 
Vom  Saal ,  1976).  Their  stereotyped  patterns  of  switching 

back  and  forth  between  alternatives  were  extremely 
persistent  and  insensitive  to  changing  contingencies.  This 
is  indicated  by  maintainance  of  the  patterns  even  when 
terminal  link  alternatives  provided  different  rates  of 
rei nforcement ,  i.e.,  FI  15-sec  versus  FI  30-sec 

alternatives,  and  when  an  extremely  long  COD  was  instituted 
with  Subject  A4.  In  the  latter  case,  switching  greatly 
increased  the  relative  time  spent  in  the  initial  links  and 
significantly  reduced  overall  earnings  for  the  session. 
This  suggests  that,  at  least  in  later  sessions,  initial  link 
responding  was  not  under  functional  control  of 
a  1 ternat i ves . 


termi na 1  link 
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Experiment  2 

Experiment  1  indirectly  compared  MI  and  AMI  schedules 
by  assessing  preference  for  each  in  comparison  to  an  FI 
schedule.  Two  subjects  preferred  MI  over  FI,  but  not  AMI 
over  FI.  Subject  A1  showed  a  transient  preference  for  MI 
over  FI,  but  no  preference  for  AMI  over  FI.  This  indicates 
that  preference  for  MI  over  FI  is  at  least  partially  a 
function  of  the  random  presentation  of  component  intervals 
in  the  MI  alternative.  It  also  suggests  that  MI  should  be 
preferred  over  AMI  in  a  direct  comparison  of  the  two 
schedules,  i.e.,  if  MI  is  preferred  over  FI  and  AMI  is  not 
preferred  over  FI,  then,  by  transitivity,  MI  should  be 
preferred  over  AMI  . 

Experiment  2  measured  preference  for  MI  30~sec  versus 
AMI  30-sec  schedules  of  monetary  reinforcement.  These 
schedules  were  presented  in  the  terminal  links  of  a 
concurrent -chai ns  procedure  similar  to  that  used  in 
Experiment  1.  Because  the  two  alternatives  differed  only  in 
the  degree  of  randomness  with  which  the  component  intervals 
were  presented,  the  present  experiment  provided  a  strong 
test  for  the  importance  of  this  factor  in  determining 
preference  for  MI.  On  the  basis  of  the  results  obtained  in 
Experiment  1,  it  was  predicted  that  subjects  would  be  more 
likely  to  prefer  the  MI  schedule  of  rei nforcement . 
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Method 

Subjects 

Four  female  undergraduates  served  as  subjects.  They 
were  selected  in  the  same  manner  as  the  subjects  in 
Experiment  1,  and  made  the  same  contractual  agreements  for 
par  t i ci pat i on . 

Apparatus  and  Setting 

The  apparatus  was  identical  to  that  used  in  Experiment 
1.  The  programming  was  changed  to  allow  the  scheduling  of 
either  FR1  or  VI  10-sec  initial  link  schedules. 

Procedure 

The  concur  rent -chai ns  procedure  was  identical  to  that 
used  in  Experiment  1.  The  sequence  of  experimental 
conditions  for  each  subject  is  outlined  in  Table  3.  In 
condition  A,  subjects  were  presented  terminal  link  schedules 
of  MI  30-sec  and  AMI  30-sec.  If  the  subject  showed  no 
schedule  preference,  condition  A2  was  instituted  in  which 
the  initial  link  schedules  were  reduced  from  VI  10-sec  to 
FR1.  Research  has  shown  that  FR1  initial  link  alternatives 
greatly  increase  the  probability  of  schedule  preference 
occurring  (Frankel  &  Vom  Saal,  1976).  If  subjects  remained 
indifferent  despite  this  manipulation,  their  further 
participation  in  the  experiment  was  terminated. 

Subjects  B3  and  B4  showed  some  preference  in  early 
sessions,  and  so  were  presented  a  more  complete  set  of 
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conditions.  Subject  B3  was  presented  an  ABA  reversal 
design,  where  B  represented  a  baseline  condition  with  AMI 
schedules  presented  in  both  terminal  links.  However, 
because  there  was  no  demonstration  of  preference  during  the 
second  presentation  of  condition  A,  initial  links  were  then 
reduced  to  FR1  to  see  if  preference  could  be  obtained. 

An  entirely  different  procedure  was  used  with  Subject 
B4.  In  place  of  a  baseline  condition,  the  position  of  the 
terminal  link  MI  and  AMI  schedules  were  simply  reversed 
across  blocks  of  sessions.  The  subject's  responding  would 
then  "track"  a  preferred  alternative  as  its  position,  left 
or  right,  changed.  This  procedure  therefore  allowed  for 
more  sessions  in  which  preference  could  be  demonstrated:  an 
important  advantage  given  the  difficulty  of  obtaining 
preference  with  previous  subjects  and  given  the  limited 
number  of  sessions  in  which  subjects  participated.  Because 
Subject  B4  developed  a  consistent  preference  over  the  first 
nine  sessions,  conditions  C  and  D  were  then  instituted. 
These  conditions  tested  whether,  by  transitivity,  the  same 
preference  pattern  would  occur  when  the  MI  and  AMI 
alternatives  were  each  compared  to  an  FI  alternative.  These 
final  conditions  were  thus  identical  to  the  comparisons  made 
i n  Exper i ment  1 . 
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Results  and  Discussion 

Resu Its 

Results  are  reported  in  the  same  manner  as  in 
Experiment  1,  and  are  based  on  data  from  entire  sessions. 
Preference  for  terminal  link  schedules  was  measured  as 
relative  rate  of  responding  on  initial  link  alternatives. 

Relative  rates  of  responding  on  initial  link 
alternatives  for  each  subject  are  presented  in  Figure  4. 
Subjects  B1  and  B2  showed  no  preference  for  either  the  MI  or 
AMI  schedules;  relative  response  rates  remained  close  to  .5 
in  all  sessions.  Since  this  pattern  occurred  even  when 
initial  link  schedules  were  reduced  to  FR1,  participation  by 
both  subjects  was  terminated  following  their  sixth  session. 

Schedule  preferences  were  shown  by  two  subjects. 
Subject  B3  seemed  to  preferred  MI  over  AMI  in  the  first  3 
sessions,  especially  in  session  3  when  70%  of  initial  link 
responding  was  on  the  alternative  leading  to  the  MI 
schedule.  In  subsequent  conditions,  however,  relative 
responding  for  the  left  alternative  remained  only  slightly 
above  .5.  This  seems  to  be  a  position  bias  since  it  also 
occurred  during  the  baseline  condition  in  sessions  4  to  6. 
When  initial  link  alternatives  were  then  reduced  to  FR1  in 
sessions  10  to  12,  a  strong  preference  for  AMI  over  MI 
developed.  Similar  results  were  obtained  with  Subject  B4 . 
While  in  sessions  2  and  3,  there  seemed  to  be  some 
preference  for  MI  over  AMI,  in  subsequent  sessions  there  was 
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strong  preference  for  the  AMI  alternative.  Greater 
preference  for  AMI  was  also  demonstrated  in  the  comparisons 
involving  the  FI  alternative;  the  subject  preferred  AMI  over 
FI  more  so  than  MI  over  FI.  In  the  latter  condition, 
exclusive  preference  for  MI  in  session  11  reverted  to  strong 
preference  for  FI  in  session  12.  On  the  other  hand, 
preference  for  AMI  over  FI  was  consistently  strong  across 
all  sessions  of  that  condition  (sessions  13  to  15). 

Additional  results  are  reported  in  Table  4.  As  in 
Experiment  1,  relative  entries  into  terminal  link 
alternatives  generally  parallel  relative  rates  of  initial 
link  responding.  When  initial  link  alternatives  are  reduced 
to  FR1,  they  are  of  course  exactly  equal.  With  respect  to 
absolute  response  rates  (not  reported  for  FR1  initial  link 
schedules),  between  subject  differences  are  apparent.  As  in 
Experiment  1,  two  extremes  were  noted  which  are  similar  to 
two  main  patterns  found  in  human  FI  performance  (Lowe, 
1979).  Extremely  high  response  rates  were  emitted  by 
Subject  B1;  extremely  low  terminal  link  response  rates  were 
emitted  by  Subjects  B3  and  B4,  with  responses  often  emitted 
only  at  the  end  of  the  scheduled  interval.  Significantly, 
these  low  rate  patterns  developed  concurrently  with  the 
development  of  subjects'  preference  for  the  AMI  alternative. 
However,  there  were  no  significant  differences  in  terminal 
link  response  rates  as  a  function  of  the  type  of  schedule  in 
operat i on . 
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Discussion 

Experiment  2  measured  preference  for  MI  versus  AMI 
schedules  of  monetary  reinforcement  presented  in  the 
terminal  links  of  a  concurrent -chai ns  procedure.  On  the 
basis  of  results  obtained  in  Experiment  1,  it  was 
hypothesized  that  subjects  would  be  more  likely  to  prefer 
the  MI  schedule  alternative.  This  would  demonstrate  that 
the  random  presentation  of  component  intervals  in  the 
variable  alternative  facilitates  preference  for  that 
a  1 ternat i ve . 

The  present  results  do  not  support  the  hypothesis,  and 
contradict  findings  obtained  in  the  previous  experiment.  In 
Experiment  1,  where  MI  and  AMI  schedules  were  each  compared 
to  an  FI  schedule,  two  subjects  showed  greater  preference 
for  MI  than  for  AMI.  In  the  present  experiment,  with  MI  and 
AMI  directly  compared,  two  subjects  preferred  AMI  more  than 
MI.  Subject  B3  showed  this  preference  only  when  the  initial 
link  schedules  were  reduced  to  FR1.  Unfortunately,  the 
effect  was  obtained  on  the  last  day  of  sessions,  and  the 
subject  was  unable  to  return  for  further  testing.  However, 
Subject  B4  demonstrated  the  effect  both  when  MI  and  AMI  were 
compared  to  each  other  and  when  each  was  compared  to  an  FI 
alternative.  Thus,  the  stronger  preference  for  AMI  was 
transitive  across  the  various  types  of  comparisons. 

Similar  to  Experiment  1,  two  subjects  in  the  present 
experiment  demonstrated  no  consistent  schedule  preferences. 
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This  pattern  was  maintained  even  when  the  initial  links  were 
reduced  to  FR1.  One  of  these  subjects,  Subject  B1,  emitted 
extremely  high  response  rates  similar  to  that  observed  in 
Subject  A4  in  Experiment  1,  who  also  showed  no  schedule 
preference.  This  suggests  that  inappropriate  high  rate 
patterns  on  interval  schedules  may  be  related  to  a  general 
insensitivity  to  experimental  contingencies  and  subsequent 
lack  of  differential  responding  to  alternatives.  Such 
insensitivity  was  certainly  the  case  with  Subject  A4,  as 
previously  noted. 

While  high  rate  patterns  are  associated  with  lack  of 
schedule  preference,  low  rate  patterns  of  terminal  link 
responding  seem  to  be  consistently  associated  with  the 
occurrence  of  schedule  preference.  In  both  experiments,  the 
strongest  preferences  were  shown  by  subjects  who  emitted 


such  patterns 

However ,  while  a 

low 

rate 

pattern 
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wi th  preference  for 

MI 
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i  n 
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it  was  associated  with 

preference 

for  AMI 

by 

two  subjects  in  the  present  experiment.  Both  Subjects  B3 
and  B4  developed  low  terminal  link  response  rates 
concomitant  with  increasing  preference  for  the  AMI  schedule. 

Interestingly,  Subject  B4  commented,  following  her 
debriefing,  that  her  preference  for  AMI  was  motivated  by  an 
attempt  to  reduce  the  effort  involved  in  obtaining 
rei nforcement .  She  stated  that  the  AMI  schedule  allowed  her 
to  know  when  to  respond.  During  the  long  component,  she 
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would  count  out  the  interval  before  responding,  while  during 
the  short  component  there  was  no  need  to  count  since 
reinforcement  was  relatively  immediate.  Thus,  she  only  had 
to  count  on  every  second  occasion  when  AMI  was  selected.  By 
contrast,  the  MI  schedule  always  required  immediate 
responding,  in  case  the  short  interval  was  upcoming,  and  the 
FI  schedule  always  required  counting  each  time  the  schedule 
was  selected.  Thus,  in  terms  of  reducing  both  overt  and 
covert  response  cost,  the  AMI  alternative  was  most 
efficient.  However,  this  explanation  implies  that  terminal 
link  response  rates  should  have  been  consistently  lower  on 
AMI  than  on  MI.  In  fact,  this  was  not  the  case;  absolute 
rates  were  equally  low  on  both  schedules,  indicating  that 
reduction  of  overt  response  cost  was  not  really  a  factor  in 
determining  preference.  Reduction  of  covert  response  cost 
does  remain  a  possible  factor,  but  unfortunately  there  is  no 
means  of  independently  assessing  the  subject's  covert 
behavior  during  schedule  performance. 

Thus,  while  Experiment  1  indicates  that  randomness  may 
be  a  contributing  factor  in  determining  preference  for 
variable  over  fixed  schedules  of  reinforcement,  Experiment  2 
indicates  that  the  opposite  may  also  be  true.  In  some 
cases,  humans  may  actually  demonstrate  decreased  preference 
for  MI  as  a  function  of  its  randomness.  Reduction  in 
response  cost  may  be  one  explanation  for  this  effect,  but 
why  the  present  results  are  opposite  to  those  obtained  in 


25 


Experiment  1  is  difficult  to  account  for.  Experiments  1  and 
2  differed  significantly  in  terms  of  the  type  and  order  of 
conditions  presented  to  subjects,  and  this  may  have  had  an 
effect.  The  inconsistent  results  may  also  be  a  function  of 
uncontrolled  for  between- sub ject  differences;  for  example, 
subjects  who  have  a  history  of  reinforcement  for  gambling 
(i.e.,  winning)  may  prefer  the  random  alternative,  while 
subjects  who  have  a  history  of  punishment  for  gambling 
(i.e.,  losing)  may  prefer  the  nonrandom  or  fixed 
alternative.  Further  research  is  required  to  determine  the 
exact  conditions  under  which  one  effect  is  obtained  versus 
the  other.  Experiment  3,  however,  examines  the  more  basic 
question  as  to  why  preference  for  variable  over  fixed 
schedules  is  so  difficult  to  obtain  in  humans,  while  it  has 
been  such  a  consistent  finding  in  animal  research. 

Experiment  3 

Results  of  Experiments  1  and  2  appear  variable  and 
contradictory .  Most  surprising  is  the  general  lack  of 
preference  for  variable  over  fixed  schedules  of 
rei nforcement ,  although  such  preference  has  been  a 
consistent  finding  in  earlier  studies  with  animals.  One 
possible  explanation  for  the  difference  is  that,  on  the 
schedules  presented,  the  human  subjects  did  not  weight  the 
i nter rei nforcement  intervals  in  the  same  manner  as  would 
animals.  More  specifically,  humans  may  be  less  effected  by 
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the  shorter  interval  to  rei nforcement  on  the  variable 
alternative,  and  may  be  more  inclined  to  respond  in  terms  of 
the  overall  rate,  rather  than  the  local  rate,  of 
reinforcement.  Thus,  given  variable  and  fixed  schedules  of 
equal  mean  length,  humans  may  tend  to  respond  equally  to 
both  alternatives.  If  a  preference  were  to  be  demonstrated, 
it  would  then  be  on  the  basis  of  some  factor  other  than  the 
local  rate  of  rei nforcement ,  for  example,  reduction  of 
response  cost. 

The  above  analysis  is  supported  by  some  informal 
observations  made  during  Experiments  1  and  2.  Following  a 
session,  subjects  would  sometimes  remark  how  boring  the 
session  was,  yet  show  considerable  interest  in  what  their 
total  earnings  were  for  the  session.  This  suggests  that 
total  rei nforcement  for  the  session  may  have  been  the  more 
potent  consequence  maintaining  behavior  during  the  session. 
That  subjects  were  specifically  instructed  to  "earn  as  much 
as  possible"  may  have  contributed  to  this  tendency.  A 
second  point  is  that,  during  informal  questioning  following 
their  participation  in  the  experiment,  some  subjects  stated 
that  they  were  well  aware  that  the  schedule  alternatives 
presented  to  them  were  generally  equal  in  terms  of  overall 
pay-off  value.  This  includes  all  subjects  who  were 
consistently  indifferent  between  the  alternatives  presented 
to  them.  It  also  includes  Subject  B4,  who  claimed  that  her 
preference  for  AMI  was  a  function  of  trying  to  reduce 
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response  cost,  as  well  as  Subject  A2 ,  who  stated  that  she 
strongly  preferred  MI  over  FI  simply  because  it  was  more 
interesting.  Thus,  these  verbal  reports  indicate  that  most 
subjects  readily  equated  the  average  payoffs  associated  with 
the  schedules  presented  to  them,  and,  in  accordance  with 
this  discrimination,  most  of  these  subjects  then  responded 
equally  to  both  alternatives. 

Herrnstein  (1981)  has  proposed  a  self-control  model 
which  is  relevant  to  the  present  discussion.  The  model 
states  that  response  strength  for  delayed  rei nforcement  may 
be  effected  by  a  number  of  factors,  including  type  of 
rei nforcement ,  level  of  deprivation,  and  inherent 
differences  between  species.  Thus,  interval  values  which 
effect  responding  for  primary  reinforcement  in  deprived 
pigeons  would  not  necessarily  be  effective  with  respect  to 
human  responding  for  secondary  rei nf orcement .  In  other 
words,  while  pigeons  may  show  strong  preference  for 
receiving  food  in  5  seconds  on  the  short  interval  of  an  MI 
schedule  versus  waiting  30  seconds  on  an  FI  schedule,  humans 
may  regard  such  a  temporal  difference  as  relatively  trivial, 
especially  when  each  reinforcement  consists  of  a  relatively 
minute  amount  (e.g.,  a  single  point  worth  only  8  cents,  as 
in  the  preceding  experiments).  Of  more  relevance  for  humans 
might  be  the  larger,  more  delayed  reinforcement  of 
maximizing  one's  total  earnings  for  the  session.  This  would 
then  account  for  the  general  pattern  of  equal  responding 
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between  alternatives  as  shown  by  many  subjects  in  the 
preceding  experiments.  It  also  suggests  that  the  use  of 
large  reinforcers  presented  at  long  delays  may  be  an 
appropriate  procedure  for  studying  certain  aspects  of  choice 
behavior  in  humans. 

Experiment  3  was  designed  as  a  demonstration  of 
possible  factors  governing  schedule  preferences  in  humans. 
Subjects  chose  between  alternative  methods  of  payment  which 
were  roughly  analogous  to  the  MI  and  FI  alternatives  used  in 
the  preceding  experiments,  only  using  much  larger  time 
delays  and  reinforcement  amounts.  Subjects  made  a  single 
choice  between  two  methods  of  payment:  the  "fixed" 
alternative  consisted  of  receiving  ten  dollars  at  the  end  of 
one  month,  while  the  "variable"  alternative  consisted  of  a 
gamble  between  receiving  the  ten  dollars  immediately  and 
receiving  it  at  the  end  of  two  months.  It  was  expected  that 
choice  between  these  alternatives  would  be  a  highly  relevant 
task  for  humans,  and  it  was  predicted  that  the  variable 
alternative  would  now  be  strongly  preferred. 

Method 

Subjects 

Ten  female  university  students  served  as  subjects. 
They  were  selected  in  the  same  manner  as  subjects  were  in 
the  previous  experiments. 
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Procedure 

The  researcher  met  with  each  subject  individually.  The 
subject  was  informed  that  the  study  required  them  to  answer 
a  couple  of  questionnaires  for  which  she  would  be  paid  ten 
dollars.  She  was  also  informed  that  she  may  not  receive  the 
money  immediately.  The  subject  was  asked  if  she  was 
agreeable  to  this  condition,  and  all  subjects  consented. 
The  researcher  next  stated  that  no  questions  would  be 
answered  while  the  questionnaires  were  being  filled  out. 
The  subject  was  then  handed  the  first  questionnaire,  and  was 
left  alone.  After  approximately  three  minutes,  the 
researcher  returned,  collected  the  first  questionnaire,  and 
handed  the  subject  the  second  questionnaire.  The  subject 
was  then  left  alone  for  approximately  five  minutes. 
Following  this,  the  subject  was  paid  the  ten  dollars, 
regardless  of  her  choice  on  the  first  questionnaire.  She 
was  then  debriefed. 

The  questionnaires  are  presented  in  Appendix  B.  On  the 
first  questionnaire,  the  subject  was  informed  that  part  of 
the  study  involved  determining  "when"  she  would  receive  the 
ten  dollars.  The  subject  was  asked  to  choose  between  two 
a  1 ternat i ves : 

1.  Alternative  A:  The  money  would  be  received  in  one 

month' s  time. 

The  money  would  be  received  either  that 
day  or  in  two  months'  time  -  to  be 


2.  Alternative  B: 
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determined  by  a  coin  flip. 

In  this  manner,  alternative  B  provided  a  50%  chance  of 
receiving  the  money  that  same  day  versus  receiving  it  in  two 
months.  With  alternative  A,  however,  it  was  a  certainty 
that  the  subject  would  have  to  wait  one  month  for  payment. 

On  the  second  questionnaire,  the  subject  was  asked  to 
indicate  her  preference  for:  (1)  receiving  the  ten  dollars 
now  versus  in  one  month,  (2)  receiving  it  in  one  month 
versus  in  two  months,  and  (3)  receiving  it  now  versus  in  two 
months.  Strength  of  preference  was  also  assessed  using 
seven  point  rating  scales.  In  addition,  the  subject  was 
asked  to  write  a  short  paragraph  on  her  reasons  for  choosing 
the  method  of  payment  that  she  did. 

Results 

On  the  first  questionnaire,  nine  out  of  10  subjects 
chose  Alternative  B,  i.e.,  to  gamble  on  receiving  the  10 
dollars  now  versus  in  two  months.  This  result  is  highly 
significant,  Z  =  +2.53,  p  <  .02.  Interestingly,  the  subject 
who  chose  not  to  gamble,  Subject  C4,  indicated  in  her 
written  response  on  the  second  questionnaire  that  she  did 
not  need  the  ten  dollars  at  that  moment,  but  would  need  it 
for  a  special  event  in  one  month. 

For  those  subjects  who  chose  Alternative  B,  responses 
to  the  second  questionnaire  were  analyzed  for  significant 
effects.  On  each  of  the  three  questions,  all  nine  subjects 
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indicated  that  they  preferred  to  receive  the  money  at  the 
earlier  period  of  time,  e.g.,  now  rather  than  in  one  month. 
On  the  strength  of  preference  ratings,  between  question 
differences  were  found  to  be  highly  significant,  F ( 2 , 16)  = 
7.44,  p  <  .01.  Further  analysis,  using  Scheffe' s  Multiple 
Range  Test,  revealed  significant  differences  between  ratings 
on  question  1  (Mean  rating  =  5.44)  and  question  2  (Mean  = 
3.44),  and  between  question  2  and  question  3  (Mean  =  5.89). 
In  other  words,  subjects  reported  significantly  stronger 
preference  for  receiving  the  money  now  rather  than  in  either 
one  or  two  months,  than  for  receiving  the  money  in  one  month 
rather  than  in  two  months.  On  the  other  hand,  subjects 
reported  that  preference  for  receiving  the  money  now  rather 
than  in  two  months  was  only  slightly  stronger  than  for 
receiving  it  now  rather  than  in  one  month.  In  summary,  the 
ratings  indicate  that  receiving  the  money  now  rather  than  in 
one  or  two  months  is  highly  preferred,  but  that  receiving  it 
in  one  month  rather  than  two  months  is  only  slightly 
prefer  red . 

The  second  questionnaire  also  required  subjects  to 
write  a  few  sentences  explaining  why  they  selected  the 
alternative  they  did  on  the  first  questionnaire.  Verbal 
reports  from  seven  out  of  the  nine  subjects,  matched  the 
differences  in  preference  ratings  noted  above.  For  example, 
Subject  C6  wrote: 

"If  I  can  wait  one  month  for  a  ten  dollar  bill,  I 
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don't  think  that  waiting  two  months  would  kill 
me.  However,  the  chance  of  receiving  it  today 
i s  more  appea ling." 

The  report  indicates  that  there  is  little  difference  between 
receiving  payment  in  one  month  rather  than  in  two  months, 
while  the  possibility  of  immediate  payment  is  highly  valued. 
It  is  also  worth  noting  that  three  of  these  subjects 
mentioned,  as  an  additional  reason,  that  they  simply 
preferred  the  "gamble".  For  example,  Subject  CIO  wrote  that 
part  of  the  reason  for  her  choice  was  "because  it's  a  bit  of 
a  gamble  and  so  more  fun,  not  just  cut  and  dried".  In  other 
words,  the  gamble  itself  was  reinforcing  which  contributed 
to  the  attractiveness  of  Alternative  B. 

Discussion 

The  results  for  Experiment  3  strongly  support  the 
hypothesis;  nine  out  of  ten  subjects  preferred  the  variable 
alternative.  The  one  subject  who  chose  the  fixed 
alternative  later  stated  that  she  did  not  need  the  money  at 
present  but  would  need  it  in  one  month.  In  behavioral 
terms,  this  might  be  referred  to  as  a  relative  lack  of 
"deprivation"  for  the  intended  reinforcer.  As  such,  her 
behavior  does  not  contradict  the  hypothesis  which  assumes 
that  subjects  have  a  present  requirement  for  the  money.  Her 
behavior  instead  seems  to  represent  a  form  of  "banking"  in 
order  to  avoid  a  future  state  of  requirement. 
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The  subjects'  ratings  indicate  that  their  selection  of 
the  variable  alternative  was  a  function  of  a  strong 
preference  for  receiving  the  money  immediately  versus  later. 
This  matches  Herrnstein's  (1964b)  notion  that  preference  for 
variable  over  fixed  schedules  is  due  to  a  heavier  weighting 
of  the  shorter  interval  to  rei nforcement .  Written 
statements  generally  parallelled  the  rating  patterns, 
although  three  subjects  indicated  that  rei nforcement 
associated  with  the  act  of  gambling  was  also  involved. 

The  variable  results  of  the  previous  experiments 
contrast  with  the  rather  consistent  results  found  in  the 
present  experiment.  Presumably,  this  is  because  the  present 
experiment  used  interval  and  rei nforcement  values  highly 
relevant  for  humans.  This  indirectly  supports  the  notion 
that  the  behavior  of  some  subjects  in  the  preceding 
experiments  was  not  being  governed  by  the  local  rate  of 
rei nforcement  available  on  each  particular  presentation  of  a 
schedule;  rather,  total  rei nforcement  for  the  session  may 
have  been  the  more  relevant  consequence.  The  present 
results  also  suggest  that,  in  order  to  obtain  results 
similar  to  those  found  in  pigeons  or  rats,  one  may  have  to 
use  parameters  which  are  in  some  ways  more  suitable  for 
humans . 

However,  it  should  be  noted  that  the  present  experiment 
differed  from  the  previous  two  experiments  in  a  variety  of 
ways.  In  addition  to  much  larger  interval  and  rei nforcement 
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values,  subjects'  choices  were  based  on  verbal  instructions 
as  opposed  to  actual  exposure  to  the  contingencies.  The 
processes  governing  behavior  under  verbal  instructions  are 
not  well  understood  at  present,  and  it  is  recognized  that 
such  behavior  may  differ  from  contingency  shaped  behavior 
(e.g.,  Catania,  Matthews,  <S  Shimoff,  1982).  In  addition, 
the  present  experiment  allowed  only  a  single  choice,  while 
preference  in  the  previous  experiments  was  measured  on  the 
basis  of  multiple  choices.  As  a  result,  the  present  study 
is  by  no  means  definitive;  it  is  simply  suggestive  as  to 
some  of  the  variables  operative  in  the  previous  two 
experiments.  Nevertheless,  it  also  stands  as  an  interesting 
finding  on  its  own,  and  is  worthy  of  further  investigation. 


General  Discussion 

The  major  purpose  of  this  thesis  was  to  investigate 
whether  humans  would  prefer  variable  over  fixed  schedules  of 
monetary  rei nforcement .  Such  preference  has  been  previously 
demonstrated  in  pigeons  responding  for  MI  versus  FI 
schedules  of  primary  reinforcement  (e.g.,  Davison,  1969). 
The  first  two  experiments  used  a  concurrent -chai ns  procedure 
similar  to  that  used  in  the  pigeon  research,  and  obtained 
mixed  results.  Experiment  3  used  analogues  of  variable  and 
fixed  schedules,  with  much  larger  time  and  reinforcement 
values,  and  obtained  relatively  consistent  preference  for 
the  variable  alternative.  The  results  suggest  that 
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preferences  in  humans  may  be  similar  to  those  found  in 
pigeons,  but  the  parameters  used  must  be  relevant  to  the 
species  being  investigated. 

In  the  preceding  discussion,  results  were  interpreted 
in  terms  of  a  strong  tendency  by  humans  to  respond  on  the 
basis  of  overall  rei nforcement  for  the  session.  Also  of 
relevance  is  the  distinction  between  w ithi n-meal  and 
bet ween -meal  behavior  as  suggested  by  Collier,  Hirsch,  and 
Kanarek  (1977).  They  propose  that  the  typical  operant 
paradigm,  with  its  emphasis  on  discrete  responses  and 
piece-meal  rei nforcement ,  is  similar  to  the  behavior  of  an 
organism  during  a  meal.  The  organism's  response  results  in 
relatively  immediate  rei nforcement ,  which  in  turn  is 
relatively  small.  As  Collier  et.  al.  point  out,  such 
within-meal  behavior  must  be  distinguished  from  between-meal 
behavior.  In  attempting  to  obtain  the  next  meal  (as  opposed 
to  the  next  bite),  the  organism  may  face  much  longer  delays 
to  reinforcement,  which  in  turn  consists  of  the  entire  meal. 
Thus,  investigations  of  within-meal  behavior  may  not  be 
useful  in  understanding  the  between-meal  behavior  of  an 
organism.  The  results  of  the  present  thesis  suggest  that 
this  problem  may  be  particularly  important  in  operant 
studies  of  human  behavior.  The  "within-meal"  paradigm  of 
the  first  two  experiments  yielded  highly  variable  results; 
the  "between-meal"  paradigm  of  the  last  experiment  yielded 
relatively  consistent  results.  Thus,  humans'  ability  to 
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respond  for  over-all  rei nforcement  in  a  session  may  mean 
that  the  be tween -mea 1  paradigm  is  a  more  appropiate  paradigm 
for  studying  certain  aspects  of  human  operant  behavior. 

Also  of  relevance  to  the  present  study  is  research  by 
Kahneman  and  Tversky  (1979)  on  risk-taking  behavior  in 
humans.  They  found  that  most  humans  tend  to  be 
" r i sk-aver se"  when  choosing  between  risky  versus  non-risky 
options  for  monetary  gain.  For  example,  if  given  a  choice 
between  a  100%  chance  of  receiving  10  dollars  and  a  50% 
chance  of  receiving  20  dollars  versus  nothing,  most  subjects 
select  the  former  nonrisky  alternative.  Both  alternatives 
are,  however,  equivalent  in  that,  if  each  were  presented  a 
number  of  times,  the  average  pay-off  (or  "expected  value") 
associated  with  each  would  be  the  same,  i.e.,  10  dollars. 
Preference  for  the  risky  alternative  is  presumed  to  be  a 
function  of  the  relative  gain  in  value  associated  with  each 
alternative.  More  specifically,  the  gain  in  value 
associated  with  an  initial  increment  of  10  dollars  is 
construed  as  greater  than  the  gain  in  value  associated  with 
any  additional  increment  of  10  dollars.  Using  Herrnstein's 
(1964b)  terminology,  the  gain  in  value  associated  with  going 
from  zero  to  10  dollars  is  "weighted  more  heavily"  than  than 
the  gain  in  value  associated  with  going  from  10  to  20 
dollars.  Thus,  receiving  10  dollars  on  the  nonrisky 
alternative  outweighs  the  50%  chance  of  receiving  an  extra 
10  dollars  on  the  risky  alternative.  The  risky  alternative 
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would  have  to  be  either  substantially  larger  in  amount 
and/or  more  probable  in  order  to  be  preferred. 

There  is  a  direct  parallel  between  the  Kahneman  and 
Tversky  (1979)  paradigm  and  the  delay  of  reward  precedure 
used  in  Experiment  3.  In  the  former  paradigm,  the 
alternatives  differ  in  the  amount  and  probability  of 
rei nforcement ;  in  the  delay  procedure,  the  alternatives 
differ  in  the  size  and  probability  of  the  interval  to 
rei nforcement .  Thus,  preference  for  the  variable 
alternative  may  be  interpreted  as  an  instance  of  risk-taking 
behavior  with  respect  to  the  temporal  delay  of 
rei nforcement .  However,  while  a  heavier  weighting  of  an 
initial  increase  in  amount  results  in  risk-averse  behavior 
in  the  Kahneman  and  Tversky  paradigm,  a  heavier  weighting  of 
the  shorter  interval  to  rei nforcement  results  in 
risk-seeking  behavior  in  the  present  procedure.  The 
principles  are  similar,  but  the  results  are  opposite. 

A  secondary  purpose  of  the  present  thesis  was  to 
investigate  the  effect  of  random  presentation  of  component 
intervals  on  preference  for  the  variable  schedule.  The 
hypothesis  in  Experiments  1  and  2  was  that  the  random 
presentation  of  intervals  should  enhance  preference  for  MI 
over  FI.  This  hypothesis  was  supported  by  the  one  subject 
who  clearly  preferred  MI  over  FI;  she  showed  no  preference 
for  AMI  over  FI.  Unfortunately,  most  subjects  showed  no 
preference  for  MI  over  FI,  and  some  subjects  even  preferred 
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the  nonrandom  AMI  alternative  over  the  MI  alternative. 
Another  problem  is  that  some  of  the  possible  processes  that 
determined  schedule  preferences  in  these  experiments  may  be 
relevant  only  to  human  behavior.  For  example,  following  her 
debriefing,  Subject  A2  reported  that  she  strongly  preferred 
MI  over  FI,  not  because  it  paid  better,  but  because  it  was 
simply  more  "interesting".  This  suggests  that  her 
preference  for  MI  may  have  been  at  least  partially  a 
function  of  secondary  rei nforcement  associated  with  the  "act 
of  gambling".  Thus,  the  results  from  these  studies  may  not 
be  that  applicable  to  previous  findings  in  pigeons,  who  are 
presumably  responding  simply  to  obtain  the  scheduled 
rei nforcer . 

While  random  presentation  of  intervals  was  not 
manipulated  in  Experiment  3,  the  results  do  provide  evidence 
on  the  importance  of  this  factor.  Subjects'  responses  to 
the  second  questionnaire  indicated  that,  with  the  exception 
of  the  "banking"  behavior  exhibited  by  one  subject,  they 
consistently  preferred  the  alternative  providing  the  shorter 
interval  to  rei nforcement .  In  addition,  the  written 
responses  by  many  subjects  indicated  that  they  selected  the 
gamble  primarily  because  of  the  possibility  of  receiving  the 
money  immediately.  All  of  this  would  suggest  that  the 
random  occurrence  of  the  shorter  interval  to  rei nforcement 
associated  with  the  variable  alternative  was  a  determining 
factor  for  the  preferences  exhibited.  However,  it  must  be 
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noted  that  these  measures  are  only  verbal  statements  of 
preference.  While  verbal  preference  may  be  suggestive  of 
behavioral  preference  under  actual  choice  conditions,  the 
two  measures  may  sometimes  yield  discrepant  results.  For 
example,  Morgan  and  Lindsley  (1966)  found  that,  in  two  out 
of  four  subjects,  verbal  preference  for  monophonic  versus 
stereophonic  music  did  not  match  actual  choice  behavior. 

A  different  point  concerns  the  insensitivity  to 
contingencies  shown  by  some  subjects  in  the  first  two 
experiments.  In  particular,  two  subjects  in  Experiment  1 
developed  stereotyped  response  patterns  which  were  extremely 
insensitive  to  procedural  changes  which  should  have 
eliminated  such  behavior.  A  post -exper iment  report  by  one 
of  these  subjects,  Subject  A4,  suggests  that  this  behavior 
was  maintained  by  the  formation  of  a  "self-rule"  during 
earlier  phases  of  the  experiment.  More  specifically,  she 
stated  that,  during  early  sessions,  she  found  that  switching 
back  and  forth  between  alternatives  was  the  best  way  to 
maximize  over-all  earnings.  In  subsequent  sessions,  she  was 
so  intent  on  following  this  strategy  that  she  did  not  notice 
changes  in  contingencies  which  made  this  strategy 
i nappropr i ate .  In  addition,  she  believed  that  points  were 
contingent  on  number  of  button  presses,  i.e.,  that  ratio 
contingencies  were  in  effect.  This  report  is  congruent  with 
her  extremely  high  "ratio- like"  response  rate. 
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Lowe 

(1979) 

has  suggested  that 

the  formation 

of 

se 1 f - ru 1 es , 

such  as 

the  above,  may 

be 

typical  of  human 

subjects  . 

On  the 

other  hand, 

Wei ner 

(  1 983 )  emphas i zes 

that 

sel f -ru les 

should 

not  be 

used 

as 

post-hoc  causal 

explanations  for 

behavior ; 

rather , 

i  t 

is  necessary 

to 

examine  the 

past  and  present 

contingencies  of  which 

the 

subject' s 

present 

verbal  behavior 

may 

be  a  by-product. 

Subject  A4' s  statement  indicates  that  her  self-rules  arose, 
at  least  partially,  as  a  function  of  her  own  interaction 
with  the  contingencies  in  earlier  phases  of  the  experiment. 
However,  Lowe  suggests  that  self-rules  may  also  arise  from 
such  factors  as  pre-exper i menta 1  history  (uncontrolled  for 
in  the  present  experiments),  and  from  instructions  provided 
by  the  exper i menter . 

The  instructional  effect  is  of  particular  relevance  to 
the  present  study.  The  instructions  used  in  Experiments  1 
and  2  emphasized  only  the  response,  i.e.,  button  pushing. 
In  a  review  of  the  literature  on  human  operant  conditioning, 
Baron  and  Galizio  (1983)  conclude  that  "instructions  about 
the  response  readily  initiate  responding  but  also  produce 
tendencies  to  respond  regardless  of  the  schedule"  (p.  500). 
This  suggests  that  the  insensitivity  to  contingencies 
exhibited  by  some  subjects  in  the  present  study  may  have 
been  to  some  extent  a  function  of  the  instructions  used. 
Thus,  self-rules  which  may  have  served  to  maintain  those 
behavior  patterns  can  be  conceptualized  as  possible 
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by-products  of  those  instructions. 

Interestingly,  Baron  and  Galizio  (1983)  note  that 
instructions  which  accurately  describe  the  contingencies,  as 
opposed  to  the  response,  tend  to  produce  response  patterns 
appropriate  to  the  schedule  of  reinforcement  in  effect.  In 
addition,  Matthews,  Shimoff,  Catania,  and  Sagvolden  (1977) 
found  that  Keeping  instructions  to  a  minimum  and  shaping  the 
response  will  also  produce  greater  sensitivity  to 
contingencies.  Thus,  if  the  present  study  had  used 
instructions  describing  the  contingencies,  or  if  responding 
had  been  shaped,  fewer  inappropriate  schedule  performances 
may  have  occurred.  In  turn,  this  may  have  eliminated  some 
of  the  inconsistent  results,  and  allowed  for  a  more  powerful 
test  of  the  hypotheses.  Further  research  is  required  to 
investigate  this  possibility. 
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Experiment  1:  Sequence  of  experimental  conditions 


Initial  Link 

Terminal  Link 

Subject 

Number  of 

C.O.D. 

Schedule 

Schedule 

(in  sec.) 

No. 

Condition 

Sessions 

(in  sec.) 

(in  sec.) 

Left 

Right 

A1 

A 

3 

1 

VI-10 

FI-30 

FI-30 

B 

II 

II 

II 

II 

MI-30 

C 

It 

II 

11 

II 

AMI-30 

B 

It 

II 

II 

II 

MI-30 

A2 

A 

3 

1 

VI-10 

FI-30 

FI-30 

C 

It 

II 

11 

II 

AMI-30 

B 

II 

1! 

11 

II 

MI-30 

C 

1! 

II 

II 

11 

AMI-30 

A3 

A 

3 

1 

VI-10 

FI-30 

FI-30 

B 

II 

II 

11 

11 

MI- 30 

C 

II 

11 

II 

11 

AMI-30 

B 

II 

If 

11 

II 

MI-30 

D 

It 

11 

II 

11 

FI-15 

A4 

A 

3 

4 

VI-10 

FI-30 

FI-30 

C 

II 

II 

It 

It 

AMI-30 

B 

II 

11 

11 

11 

MI-30 

C 

1 

It 

11 

11 

AMI- 30 

Dl 

1 

11 

It 

11 

FI-15 

D2 

1 

15 

1 1 

II 

FI-15 

Experiment  1:  Relative  rate  of  responding  in  initial  links,  relative  entries  into  terminal  links,  number  of  entries  into  each  terminal  link,  and 
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Table  3 


Experiment  2:  Sequence  of  experimental  conditions 


Initial 

Terminal  Link 

Number  of 

Link 

Schedule 

(in  sec.) 

Subject  ( 

Condition 

Sessions 

Schedule 

Left 

Right 

B1  &  B2 

A1 

3 

VI-10" 

MI-30 

AMI-30 

A2 

ft 

FR-1 

ft 

ft 

B3 

A1 

3 

VI-10" 

MI-30 

AMI-30 

B 

ft 

»T 

AMI-30 

ft 

A1 

ft 

f  1 

MI-30 

ft 

A2 

ft 

FR-1 

ft 

ft 

B4 

A 

3 

VI-10" 

MI-30 

AMI- 30 

B 

ft 

ft 

AMI-30 

MI-30 

A 

ft 

ft 

MI-30 

AMI-30 

C 

ft 

ft 

t! 

FI-30 

D 

ft 

ft 

AMI-30 

ft 

Note:  The 

experiment 

was  terminated  early 

for  Subjects 

B1  and  B2 

due 

to  indifference  between 

l  terminal 

link  alternatives. 

For 

Subject  B3 

,  preference 

was  measured  in  comparison  to  a 

baseline  phase;  for  Subject  B4,  preference  was  measured,  in 
the  first  nine  sessions,  by  alternating  the  positions  of  the 
MI  and  AMI  terminal  link  schedules. 
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the  terminal  link  and  consumatory  response  buttons  were  blue  and  yellow. 
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Figure  2:  Diagram  of  procedure  in  Experiment  1.  In  the  initial  links,  both  alternatives  are  available, 
the  terminal  links  and  consumatory  response  phase,  only  one  alternative  is  available. 


Right  Terminal  Link  Schedule  (Left  =  FI  30)  Right  Terminal  Link  Schedule  (Left  =  FI  30) 
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Left  Terminal  Link  Schedule  (Right  =  AMI  30) 


51 


CN 


(i  +  a  / 1) 

6u|puodsay  to  apy  aAjppy 


c 

o 

co 

CO 

<D 

C/D 


W  o 

0)  CO 

■D  2 

CD 

jr 

<8  8 


re 

c 


§  I 

<D  < 


m 

CM 


—  O 


a 

oo 

>0 

co 

csi 


6ui.puodsay  p  apy  aAjppy 


Session  # 

Figure  4:  Relative  rate  of  initial  link  responding  to  left  alternative  for  each  subject  in  Experiment 
^Initial  link  schedules  reduced  from  VI-10-sec.  to  FR-1  for  these  sessions. 


References 


Autor,  S.M.  The  strength  of  conditioned  rei nforcers  as  a 
function  of  frequency  and  probability  of  r i enforcement . 
Doctoral  dissertation,  Harvard  University,  1960. 
Published  in  D.P.  Hendry  (Ed.),  Conditioned 

reinforcement.  Homewood,  Ill.:  Dorsey,  1969. 

Baron,  A.,  &  Galizio,  M .  Instructional  control  of  human 
operant  behavior.  The  Psychological  Record,  1983,  33, 
495-520. 

Catania,  A.C.,  Matthews,  B.A.,  <§  Shimoff,  E.  Instructed 
versus  shaped  human  verbal  behavior:  Interactions  with 
nonverbal  responding.  Journal  of  the  Experimental 
Analysis  of  Behavior,  1982,  83,  233-248. 

Cicerone,  R.A.  Preference  for  mixed  versus  constant  delay 
of  reinforcement.  Journal  of  the  Exper i mental  Analysis 
of  Behavior,  1976,  25,  257-261. 

Collier,  G.,  Hirsch,  E.,  <§  KanareK,  R.  The  operant 

revisited.  In  W.K.  Honig  <&  J.E.R.  Staddon  (Eds.), 
Handbook  of  operant  behavior.  New  York:  Prentice  Hall, 
1977. 

Davison,  M.  Preference  for  mi xed- i nterva 1  versus 
f i xed- i nterva 1  schedules.  Journal  of  the  Experimental 
Analysis  of  Behavior,  1969,  12,  247-252. 

Davison,  M.  Preference  for  mi xed- i nterva 1  versus 
f i xed- i nterva 1  schedules:  Number  of  component 


52 


' 


53 


intervals.  Journal  of  the  Experimental  Analysis  of 
Behavior,  1972,  17,  169-176. 

de  Villiers,  P.A.  Choice  in  concurrent  schedules  and  a 
quantitative  formulation  of  the  law  of  effect.  In  W.K. 
Honig  &  J.E.R.  St addon  (Eds.),  Handbook  of  operant 
behavior.  New  York:  Prentice  Hall,  1977. 

Emerson,  R.M.  Exchange  theory,  part  I:  A  psychological 
basis  for  social  exchange.  In  J.  Berger,  M. 

Zelditch,  Jr.,  &  B.  Anderson  (Eds.),  Sociological 
Theories  in  Progress  (Vol.  2).  Boston:  Houghton 
Miffin,  1972. 

Fantino,  E.  Preference  for  mixed-  versus  fixed-ratio 
schedules.  Journal  of  the  Experimental  Analyis  of 
Behavior,  1967,  10,  35-43. 

Fantino,  E.,  &  Moore,  J.  Uncertainty  reduction,  conditioned 
reinforcement,  and  observing.  Journal  of  the 

Experimental  Analyis  of  Behavior,  1980,  33,  3-13. 

Findley,  J.  Preference  and  switching  under  concurrent 

scheduling.  Journal  of  the  Experimental  Analyis  of 
Behavior,  1958,  1,  123-144. 

Frankel ,  P.W.,  &  Vom  Saal ,  W.  Preference  between 

f i xed- i nterva 1  and  var i ab 1 e- i nterva 1  schedules  of 
rei nforcement :  Separate  roles  of  temporal  scaling  and 
predictability.  Animal  Learn i ng  &  Behav i or ,  1976,  4, 
71-76. 

Green,  L.  Preference  as  a  function  of  the  correlation 


- 


54 


between  stimuli  and  reinforcement  outcomes.  Learning 
and  Mot i vat  ion ,  1980,  11,  238-255. 

Hendry,  D.P.  Reinforcing  value  of  information:  Fixed-ratio 
schedules.  In  D.P.  Hendry  (Ed.),  Conditioned 
reinforcement.  Homewood,  Illinois:  Dorsey  Press,  1969. 

Herrnstein,  R.J.  Relative  and  absolute  strength  of  response 
as  a  function  of  frequency  of  reinforcement.  Journal 
of  the  Exper i menta 1  Anal ys i s  of  Behav ior ,  1961,  4 , 

267-272. 

Herrnstein,  R.J.  Secondary  reinforcement  and  rate  of 

primary  rei nforcement .  Journal  of  the  Experimental 
Analysis  of  Behavior,  1964,  7,  27-36. (a) 

Herrnstein,  R.J.  Aperiodicity  as  a  factor  in  choice. 
Journal  of  the  Experimental  Analysis  of  Behavior,  1964, 
7,  179-182. (b) 

Herrnstein,  R.J.  Self-control  as  response  strength.  In 
C.M.  Bradshaw  (Ed.),  Recent  developments  in  the 

quant  if icat ion  of  steady  state  operant  behavior, 
Amsterdam:  E 1 sevier/Nor th-Hol 1  and  Biomedical  Press, 

1981  . 

Hursh,  S.R.,  &  Fantino,  E.  Relative  delay  of  rei nforcement 
and  choice.  Journal  of  the  Experimental  Analysis  of 
Behavior,  1973,  19,  437-450. 

Hursh,  S.R.,  &  Fantino,  E.  An  appraisal  of  preference  for 

multiple  versus  mixed  schedules.  Journal  of  the 

Exper i mental  Analysis  of  Behavior,  1974,  22,  31-38. 


' 


55 


Kahneman,  D.,  &  TversKy,  A.  Prospect  theory:  An  analysis  of 
decision  under  risk.  Econometrics,  1979,  47,  263-291. 

Killeen,  P.  On  the  measurement  of  reinforcement  frequency 
in  the  study  of  preference.  Journal  of  the 
Experimental  Analysis  of  Behavior,  1968,  11,  263-269. 

Lowe,  C.F.  Determinants  of  human  operant  behavior.  In  M.D. 
Zeiler  &  P.  Harzem  (Eds.),  Advances  in  analysis  of 
behavior(\J ol  .  1).  Reinforcement  and  organization  of 

behavior.  New  York:  Wiley,  1979. 

Matthews,  B.A.,  Shimhoff,  E.,  Catania,  C.,  &  Sagvolden,  T. 
Uninstructed  human  responding:  Sensitivity  to  ratio  and 
interval  contingencies.  Journal  of  the  Experimental 
Analysis  of  Behavior,  1977,  27,  453-467. 

Morgan.  B.J.,  &  Lindsley,  O.R.  Operant  preference  for 

stereophonic  over  monophonic  music.  Journal  of  Music 
Therapy,  1966,  3,  135-144. 

Navarick,  D.J.,  &  Fantino,  E.  Transitivity  as  a  property  of 
choice.  Journal  of  the  Exper i mental  Analysis  of 
Behavior,  1972,  18,  389-401. 

Navarick,  D . 0  .  ,  &  Fantino,  E.  Stochastic  transitivity  and 

un i d i mens i ona 1  control  of  choice.  Learning  and 
Motivation,  1975,  6,  179-201.' 

Pierce,  W.D.,  &  Epling,  W.F.  Choice,  matching,  and  human 
behavior:  A  review  of  the  literature.  The  Behavior 

Analyst,  1983,  6,  57-76. 

Repp,  A.C.,  &  Deitz,  S.M.  A  comparison  of  fixed-ratio  and 


' 


56 


variable-ratio  token-production  schedules  with  human 
subjects.  The  Psychol og ical  Record ,  1975,  25,  131-137. 

Rider,  D.P.  Preference  for  mixed  versus  constant  delays  of 
rei nforcement :  Effect  of  probability  of  the  short, 
mixed  delay.  Journal  of  the  Experimental  Analysis  of 
Behavior ,  1983,  39,  257-266. (a) 

Rider,  D.P.  Choice  for  aperiodic  versus  periodic  ratio 
schedules:  A  comparison  of  concurrent  and 

concurrent-chains  procedures.  Journal  of  the 
Experimental  Analysis  of  Behavior,  1983,  40, 

225-237. (b) 

Sherman,  J.A.,  &  Thomas,  J.R.  Some  factors  controlling 
preference  between  fixed-ratio  and  variable-ratio 
schedules  of  rei nforcement .  Journal  of  the 

Experimental  Analysis  of  Behavior,  1968,  11,  689-702. 

Weiner,  H.  Preference  and  switching  under  ratio 

contingencies  with  humans.  Psychological  Reports, 
1966,  18,  239-246. 

Weiner,  H.  Some  thoughts  on  discrepant  human-animal 

performances  under  schedules  of  rei nforcement . 
Psychological  Records,  1983,  33,  521-532. 


Appendix  A 


Preference  for  Variable  versus  Fixed  Schedules  of 
Reinforcement:  A  Review  of  the  Literature 

Schedules  of  reinforcement  may  be  categorized  in 
several  ways.  The  most  basic  distinctions  are  between  ratio 
and  interval  schedules,  and  between  variable  and  fixed 
schedules.  The  present  paper  reviews  research  which  has 
generally  shown  that  organisms'  prefer  variable  over  fixed 


schedules 

of 

rei nforcement , 

a  1  though 

the 

mean 

rate  of 

rei nforcement 

on 

both  schedules  may 

be 

equa 1 , 

First, 

however , 

it 

may 

be  useful 

to  outline 

the 

basic 

paradigms 

used  in  this  research. 

Preference  Paradigms 

The  studies  in  this  review  have  utilized  two  different 
paradigms  for  determining  schedule  preferences.  The  most 
common  paradigm  is  the  concur  rent -cha i ns  procedure  as  first 
suggested  by  Autor  (1969).  In  the  initial  links  of  the 
chain,  the  organism  is  presented  with  two  stimuli,  both 
associated  with  equal  var i ab 1 e- i nterva 1  (VI)  schedules. 
Meeting  the  requirement  on  either  alternative  is  reinforced 
by  production  of  a  second  stimulus  associated  with  another 
schedule  of  rei nforcement .  Meeting  the  requirement  on  this 
terminal  link  schedule  then  results  in  primary 
reinforcement.  When  a  terminal  link  schedule  is  in  effect, 


57 


58 


the  other  alternative  becomes  inoperative.  Thus,  during  the 
initial  links  of  the  chain,  the  organism  chooses  between  two 
mutually  exclusive  terminal  link  schedules,  for  example,  a 
fixed-interval  (FI)  schedule  and  a  VI  schedule.  Preference 
for  terminal  link  schedules  is  typically  measured  by 
relative  rate  of  responding  on  the  initial  link  schedules. 

An  advantage  of  the  concur  rent -chai ns  procedure  is  that 
it  separates  rate  of  responding  for  a  schedule  (i.e., 
initial  link  responding)  from  rate  of  responding  on  a 
schedule  (i.e.,  terminal  link  responding).  Rate  of 
responding  on  a  schedule  is  invalid  as  a  measure  of 
preference  since  it  may  be  influenced  by  character i st i cs  of 
the  schedule  itself.  For  example,  ratio  schedules  typically 
generate  higher  response  rates  than  interval  schedules, 
because  ratio  schedules  provide  intermittent  rei nforcement 
for  shorter  i nter response  times  (Ferster  &  Skinner,  1957). 
Thus,  higher  response  rates  on  ratio  as  opposed  to  interval 
schedules  may  be  a  function  of  schedule  char acter i s t i cs ,  and 
do  not  necessarily  indicate  preference  for  that  schedule. 
An  analogous  situation  would  be  to  incorrectly  assume  that 
cleaning  house  is  preferred  to  watching  television,  because 
the  former  has  a  higher  rate  of  activity. 

A  second  paradigm,  which  has  been  employed  to  measure 
preference  for  variable  ratio  (VR)  versus  fixed  ratio  (FR) 
schedules,  is  the  switching  procedure  devised  by  Findley 
(1958).  Responses  on  a  change-over  (CO)  alternative  change 
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or  switch  the  schedule  of  rei nforcement  in  effect  on  a 
second  alternative.  Preference  is  measured  as  the  relative 
frequency  of  change-overs  from  one  schedule  to  another.  For 
example,  if  the  procedure  is  arranged  so  that  an  FR  schedule 
is  presented  at  the  start  of  each  trial,  but  the  subject 
continually  switches  to  an  alternative  VR  schedule,  then  one 
can  conclude  that  the  subject  prefers  VR  over  FR.  Similar 
to  the  concurrent -chai ns  procedure,  this  procedure  separates 
responding  for  a  schedule  (i.e.,  change-over  responding) 
from  responding  on  a  schedule.  In  addition,  Findley  has 
shown  that  the  degree  of  preference  can  be  manipulated  by 
changing  the  response  requirement  on  the  CO  alternative. 
Increasing  the  requirement  tends  to  reduce  the  amount  of 
switching  from  one  schedule  to  another,  and  increases 
preference  for  the  schedule  presently  in  effect. 

Preference  for  VI  versus  FI  Schedules 

All  of  the  research  on  preference  for  VI  versus  FI 
schedules  has  used  the  concurrent-chains  paradigm.  In  the 
first  study  to  be  described,  this  procedure  will  be  outlined 
in  some  detail  to  facilitate  clarity.  Following  this, 
discussions  of  the  other  studies  will  emphasize  only  those 
differences  from  the  basic  procedure  which  may  be 
s i gni f i cant . 

Preference  for  VI  versus  FI  schedules  was  first 
investigated  by  Herrnstein  (1964b).  In  the  initial  links  of 
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a  concurrent -cha i ns  procedure,  pigeons  chose  between  two 
response  Keys,  both  tr ans i 1 1 umi nated  with  a  white  light. 
Both  Keys  were  programmed  on  independent  VI  60- sec 
schedules.  Meeting  the  schedule  requirement  on  the  right 
Key  changed  the  Key  color  to  yellow,  and  resulted  in  two 
successive  presentations  of  a  terminal  1 i nK  FI  schedule. 
Responding  on  the  FI  schedule  was  reinforced  by  three 
seconds  access  to  grain.  A  similar  sequence  was  arranged  on 
the  left  Key,  except  that  the  terminal  linK  sometimes 
consisted  of  VI  schedules.  The  left/right  terminal  linK 
schedules  in  each  condition  were  VI  15-sec/FI  15-sec,  VI 
15-sec/FI  4-sec,  VI  15-sec/FI  8-sec,  and  FI  15-sec/FI  4-sec 
in  that  order.  Earlier  research  had  shown  that  pigeons 
would  match  rate  of  responding  of  initial  linK  alternatives 
to  rate  of  primary  rei nforcement  on  the  terminal  linK 
alternatives,  when  both  terminal  1 i nKs  consisted  of  VI 
and/or  VR  schedules  (Autor,  1969;  Flerrnstein,  1964a).  In 
comparing  the  VI  and  FI  schedules,  however,  Flerrnstein  found 
a  consistent  bias  for  the  VI  alternative.  Thus,  even  when 
the  arithmetic  means  of  both  schedules  were  equal,  i.e.,  VI 
15-sec/FI  15-sec,  the  four  birds  emitted  over  70%  of  their 
initial  linK  responses  on  the  Key  leading  to  the  VI 
alternative.  If  subjects  had  been  matching  rate  of 
responding  to  rate  of  rei nforcement ,  responding  should  have 
been  equally  distributed  between  the  two  alternatives. 
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To  account  for  the  bias  towards  the  VI  alternative, 
Herrnstein  (1964b)  suggests  that  pigeons  may  "weight  the 
shorter  intervals  of  the  variable  interval  schedule  more 
than  the  longer"  (p.  181).  Thus,  the  assumption  of 
arithmetic  averaging  of  intervals  in  VI  schedules  may  be  an 
i nappropr i ate  description  of  their  value  relative  to  FI 
schedules.  Because  the  shorter  i nter rei nforcement  intervals 
in  the  VI  schedule  may  exert  a  di spropor t i onate  influence  on 
responding,  a  VI  schedule  is  not  equivalent  to  an  FI 
schedule  with  the  same  arithmetic  mean.  Herrnstein  also 
suggests  that  it  may  be  possible  to  derive  a  tr ansformat i on 
rule  for  interval  values  which  would  predict  the  observed 
preferences,  but  found  that  a  logarithmic  transformat  ion  was 
inadequate.  (A  logarithmic  transformat  ion  results  in  a 
geometric  mean  of  interval  values  which  weights  shorter 
intervals  more  heavily.)  Finally,  Herrnstein  notes  the 
similarity  of  the  results  to  gambling  behavior  in  humans. 
In  gambling,  the  slight  possibility  of  a  relatively 
immediate  gain  is  highly  attractive  to  many  individuals. 
They  prefer  to  gamble  rather  than  invest  their  money  in  more 
conservative  ventures  which,  though  more  certain,  pay  off 
only  after  a  much  longer  interval. 

Killeen  (1968)  attempted  to  derive  a  specific 
transformat  ion  rule  which  would  describe  preference  for  VI 
over  FI.  In  Experiment  1,  the  terminal  link  alternatives 
consisted  of  single  presentations  of  a  VI  schedule  and  an  FI 
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schedule  of  primary  reinforcement.  For  VI  23-sec,  54-sec, 
and  31 -sec  schedules,  in  that  order,  FI  schedules  were  found 
which  resulted  in  equal  responding  for  both  alternatives. 
For  the  group  data,  a  transformation  rule  was  then  derived 
which  would  predict  such  equivalence.  The  transformation 
was  based  on  a  power  function  of  the  form: 

N 

M(yr)  =  l/N  L  y£ 

i  =  1 

where  M(yr)  is  the  mean  interval  of  the  schedule,  y^is  the 
value  of  the  i th  component  interval  of  the  schedule,  N  is 
the  number  of  components,  and  r  is  a  parameter.  As  r 
becomes  more  negative,  shorter  component  intervals  are 
weighted  more  heavily.  Killeen  found  that  a  harmonic 
transformation  of  the  intervals  (r=-1),  i.e.,  a  reciprocal 
transformation  of  the  intervals,  was  most  appropriate  to 
describe  his  results.  In  other  words,  the  pigeons  were 
indifferent  between  VI  and  FI  schedules  with  equal  harmonic 
means,  although  the  FI  schedule  always  had  a  smaller 
arithmetic  mean.  This  transformation  seemed  to  account  for 
Herrnstein' s  (1964b)  data  also.  Experiment  2  provided 
further  support  for  the  validity  of  the  harmonic 
transformation.  Both  terminal  link  alternatives  were  VI 
schedules.  The  number  and  size  of  the  short  versus  long 
component  intervals  in  each  schedule  was  varied  so  that  one 
schedule  had  a  longer  arithmetic  mean  but  a  shorter  harmonic 
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mean  than  the  other.  As  expected,  relative  rate  of 
responding  in  the  initial  links  matched  the  harmonic  rate 
and  not  the  arithmetic  rate  of  rei nforcement . 

Davison  (1969)  examined  preference  for  FI  versus 
mi xed- i nterva 1  (MI)  schedules  of  reinforcement.  (A  mixed 
schedule  can  be  considered  as  a  type  of  VI  schedule  which 
controls  for  the  number  and  length  of  component  values.  For 
example,  a  two-value  MI  30-sec  schedule  may  consist  of  15- 
and  45-sec  intervals  presented  in  random  order .  This  can 
also  be  referred  to  as  a  "mixed  FI  15-sec  FI  45-sec" 
schedule.)-  The  concur  rent -chai ns  procedure  was  similar  to 
Killeen's  (1968)  except  that  the  initial  link  alternatives 
were  separated  by  a  0.5-sec  change-over  delay  (COD).  [A  COD 
is  commonly  used  in  concurrent  schedules  to  prevent 
adventitious  rei nforcement  of  switching  behavior,  but  is  not 
commonly  employed  in  the  concur rent -cha i ns  procedure 
(Catania,  1966).]  Across  conditions,  an  MI  30-sec  schedule, 
consisting  of  15-  and  45-sec  components,  was  compared  to 
30- ,  1  0 - ,  2 0 - ,  1 5 - ,  and  25-sec  FI  schedules,  in  that  order . 
For  both  individual  birds  and  group  data  (N=5),  Davison 
found  that  the  reciprocals  of  the  intervals  transformed  to 
the  third  power  (r=- 3)  best  described  the  data.  This  cubic 
transformat i on  weights  the  shorter  intervals  more  heavily 
than  does  the  harmonic  transformat  ion  suggested  by  Killeen. 

Davison  (1972)  investigated  whether  number  of  component 
intervals  would  effect  preference  for  MI  versus  FI.  The 
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procedure  was  basically  identical  to  Davison  (1969),  but 
with  no  COD  in  the  initial  links.  Various  combinations  of 
FI  schedules,  ranging  in  value  from  10-  to  30-sec,  were 
compared  with  MI  30-sec  schedules  containing  either  two, 
three,  or  seven  component  intervals.  In  each  of  these  MI 
schedules,  the  shortest  and  longest  components  were  15-  and 
45-sec  respectively.  The  results  from  six  pigeons  indicated 
that  the  number  of  component  intervals  did  not  effect 
preference.  This  also  means  that  the  number  of  times  the 
shortest  interval  was  presented  also  had  no  effect  on 
preference.  Davison  found  that  preference  in  all  conditions 
was  best  described  by  the  mean  of  the  reciprocals  of  the 
intervals  squared  ( r  =  -  2 ) .  This  inverse  square 
transformation  weights  the  shorter  intervals  more  than 
Killeen's  (1968)  harmonic  transformat  ion ,  but  less  than 
Davison's  (1969)  cubic  transformat  ion .  In  accounting  for 
Killeen's  data,  Davison  notes  that  Killeen  used  VI  schedules 
which  contained  extremely  short  component  intervals  (3-sec) 
compared  to  the  15-sec  interval  in  his  own  MI  schedule. 
Thus,  the  length  of  the  shortest  interval  may  be  a  critical 
factor  in  determining  the  proper  transformation;  with 
extremely  short  components,  a  harrrpnic  transformat  ion  may  be 
most  appropriate.  Davison  also  suggests  that  the  cubic 
transformat  ion  obtained  in  his  earlier  study  may  be  due  to 
the  use  of  a  COD  procedure.  For  example,  Shull  and  Pliskoff 
(1967)  found  that,  with  concurrent  schedules,  a  COD 
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increases  preference  for  the  alternative  providing  the 
greater  rate  of  rei nforcement .  Similarly,  in  the  Davison 
(1969)  study,  the  COD  may  have  enhanced  preference  for  the 
MI  alternative,  thereby  altering  the  appropriate 
transformat i on . 

Hursh  and  Fantino  (1973,  Experiment  1)  used  a 
concurrent -chai ns  procedure  to  investigate  preference  for  MI 
versus  FI  when  the  shortest  interval  in  the  MI  alternative 
was  varied.  Across  conditions,  the  FI  schedule  varied 
between  10-  and  50-sec,  while  the  MI  schedule  consisted  of  a 
long  interval  of  60-sec  and  a  short  interval  of  either  1 0 - , 
20- ,  or  30-sec.  It  was  predicted  that  the  appropriate 
transformation  rule  would  vary  as  the  shortest  interval  in 
the  MI  schedule  varied.  This  hypothesis,  however,  was  not 
supported.  Across  all  conditions,  the  data  was  best 
described  by  Davison's  (1972)  inverse  square  transformation 
(r=-2).  In  fact,  this  tr ansformat i on  accounted  for  91%  and 
96%  of  the  variance  for  individual  birds,  and  93.5%  of  the 
group  variance.  Hursh  and  Fantino  conclude  that  the  inverse 
square  transformation  appears  to  be  quite  general.  They  do 
suggest,  however,  that  for  VI  schedules  with  very  short 
components,  as  in  Killeen  ,(1968),  the  harmonic 
tr ansformat i on  may  be  more  appropriate. 

NavaricK  and  Fantino  (1972)  tested  whether  preference 
for  terminal  link  VI  versus  FI  schedules  would  be 
"transitive"  to  comparisons  with  a  third  schedule.  In  one 
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procedure,  FI  schedules  were  found  which  were  equally 
preferred  to  VI  23-  and  54-sec  schedules.  These  VI  and  FI 
schedules  were  then  each  compared  to  an  FI  20-sec  schedule. 
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"Strong  stochastic  transitivity"  would  hold  if  the 
equivalent  VI  and  FI  schedules  were  equally  preferred  to  the 
FI  20-sec  schedule.  In  tests  involving  the  VI  23-sec 
schedule,  transitivity  held  for  three  out  of  four  birds; 
however,  zero  out  of  three  birds  demonstrated  transitivity 
in  tests  involving  the  VI  54-sec  schedule.  A  similar  result 
was  obtained  with  a  second  procedure:  VI  and  FI  schedules 
which  were  equally  preferred  over  a  standard  FI  schedule 
were  not  always  equivalent  when  compared  to  each  other. 
Such  intransitivity  was  also  found  in  comparisons  involving 
VR  and  FR  schedules.  NavaricK  and  Fantino  suggest  that  the 
results  indicate  that  variable  schedules  are  substantially 
different  from  fixed  schedules,  and  that  a  single  general 
transformation  rule  to  equate  VI  and  FI  schedules  may  not  be 
possible  (see  also  NavaricK  &  Fant i no , 1 975 ) . 

Schrader  and  Rachlin  (1976)  examined  the  effect  of 
signaled  reinforcement  on  preference  for  VI  versus  FI.  The 
values  for  both  VI  and  FI  schedules  were  30-sec,  15-sec, 
6-sec,  and  2.5-min  in  that  order.,  To  control  for  length  of 
terminal  links,  the  VI  terminal  link  was  always  equal  in 
length  to  the  FI  terminal  link,  and  the  number  of  VI 
reinforcements  varied  randomly  between  zero  and  two.  In  the 
signaled  condition,  the  occurrence  of  rei nforcement  in  each 
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terminal  link  was  preceded  by  a  brief  change  in  key  color. 
While  previous  research  had  shown  that  rats  prefer  signaled 
over  unsignaled  shock  (Badia,  Harsh,  &  Coker,  1975), 
Schrader  and  Rachlin  found  no  effect  of  signaled 
reinforcement  on  preference  for  VI  over  FI  schedules.  They 
had  predicted  that  the  signal  would  effect  preference  by 
eliminating  the  differences  between  schedules  in 
predictability  of  rei nforcement .  However,  the  signal  did 
influence  terminal  link  responding;  response  rates  preceding 
the  signal  were  relatively  slow,  but  increased  considerably 
when  the  signal  was  presented.  In  the  unsignalled 
conditions,  response  rates  varied  little  throughout  the 
intervals,  especially  for  the  VI  schedules.  Interestingly, 
as  the  values  of  the  VI  and  FI  schedules  decreased, 
preference  for  the  VI  schedule  also  decreased.  For  example, 
three  out  of  four  birds  emitted  more  than  90%  of  their 
initial  link  responses  to  the  key  leading  to  the  VI  schedule 
when  the  mean  schedule  values  were  30-sec  or  greater.  With 
the  6- sec  schedules,  however,  responding  for  the  VI 
alternative  dropped  to  between  60%  and  70%.  This  effect 
would  be  expected  on  the  basis  of  a  heavier  weighting  of 
shorter  intervals;  the  short  component  in  the  VI  schedule, 
e.g.,  3-sec,  would  be  that  much  shorter  than  the  FI  schedule 
interval  as  the  values  of  these  schedules  increased,  e.g., 
6-sec  to  30-sec.  This  assumes  that  the  short  component  in 
the  VI  alternative  remained  relatively  constant  as  the  mean 
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length  of  the  schedule  increased.  Unfortunately,  Schrader 
and  Rachlin  do  not  report  what  the  VI  components  were. 

Frankel  and  Vom  Saal  (1976)  examined  the  effect  of 
interval  "predictability"  on  preference  for  MI  versus  FI. 
Their  research  was  based  on  an  earlier  study  by  Bower, 
MacLean,  and  Meacham  (1966)  who  found  that  pigeons  preferred 
mu  1 1 i p 1 e- i nterva 1  over  mi xed- i nterva 1  schedules.  (A 
multiple  schedule  differs  from  a  mixed  schedule  in  that  each 
component  is  accompanied  by  a  distinctive  stimulus.)  Thus, 
pigeons  preferred  the  terminal  link  where  interval  values 
were  made  predictable  by  correlated,  as  opposed  to 
uncor re  1 ated ,  terminal  link  key  colors  (see  also  Fantino  & 
Moore,  1980;  Green,  1980;  and  Hursh  &  Fantino,  1974).  On 
the  basis  of  this  result,  Frankel  and  Vom  Saal  predicted 
that  pigeons  would  prefer  a  mu  1 1 i p 1 e- i nterva 1  over  an  FI 
schedule  more  so  than  an  MI  over  an  FI  schedule.  Using  a 
concur  rent -chai ns  procedure  with  VI  60-sec  initial  links, 
there  was  a  slight  but  consistent  effect  in  the  predicted 
direction  for  all  seven  birds.  They  also  replicated  the 
Bower  et  al.  finding,  but  only  when  the  initial  links  were 
reduced  from  VI  60-sec  to  FR1  as  used  in  the  original  study. 
This  last  finding  concurs  with  Hursh  and  Fantino  (1974)  who 
note  that  decreasing  the  length  of  the  initial  link 
schedules  will  increase  the  preference  shown  for  one 
terminal  link  schedule  over  the  other. 
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Preference  for  VR  versus  FR  Schedules 

Fantino  (1967)  used  a  concur rent -cha i ns  procedure  to 
examine  preference  for  mixed-ratio  (MR)  versus  FR  schedules 
of  rei nforcement .  The  range  of  the  MR  schedule  components 
was  manipulated  (the  short/long  intervals  were  either  1/99, 
10/90,  or  25/75),  as  well  as  the  number  of  MR  components  - 
either  two  (1/99)  or  three  (1/50/99).  All  five  pigeons 
preferred  MR  over  FR  schedules  of  equal  mean  value.  In 
addition,  preference  for  MR  generally  increased  as  the  range 
in  component  values  was  increased;  relative  rate  of 
responding  for  MR  was  sightly  below  60%  with  the  25/75 
components  and  above  70%  with  the  1/90  components.  The 
number  of  component  values  in  the  MR  schedule  had  no  effect 
on  preference,  matching  Davison's  (1972)  results  with  MI 
versus  FI  schedules.  Finally,  Fantino  determined  that 
relative  rate  of  responding  in  the  initial  links  was  closely 
approximated  by  the  relative  geometric  rates  of 
reinforcement  in  the  terminal  links. 

As  part  of  a  series  of  experiments  on  preference  for 
informational  stimuli,  Hendry  (1969)  compared  a  VR  schedule 
and  two-value  multiple-  and  mixed-ratio  schedules  with 
various  FR  schedules.  A  concurrent -chai ns  procedure  was 
used  with  FRIO  initial  links.  Unlike  Frankel  and  Vom  Saal's 
(1976)  results  with  MI  schedules,  discussed  previously, 
Hendry  found  that  preference  over  FR  was  not  reliably 
greater  for  the  multiple  schedule  than  for  the  mixed 
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schedule.  The  multiple  schedule  was,  however,  preferred 
over  FR  more  so  than  VR  was  preferred  over  FR.  In  addition, 
the  multiple  schedule  was  strongly  preferred  over  the  mixed 
schedule  in  a  direct  comparison  of  the  two.  Hendry 
interprets  preference  for  the  multiple  schedule  in  terms  of 
the  reduction  of  "uncertainty";  the  multiple  schedule's  Key 
colors  were  correlated  with  the  component  values  of  the 
schedule  which  were  thereby  made  more  predictable.  Hendry 
also  attempted  to  derive  a  transformat  ion  rule  to  describe 
his  data,  but  found  that  a  harmonic  transformation  of 
reinforcement  rates  was  generally  inadequate. 

Sherman  and  Thomas  (1968)  used  a  switching  procedure  to 
examine  preference  between  nine  FR  schedules  with  correlated 
stimuli  (i.e.,  a  ni ne- component  multiple-ratio  schedule)  and 
an  MR  schedule  consisting  of  the  same  nine  values  presented 
with  the  same  stimulus.  The  schedule  values  were  1,  30,  60, 
90,  120,  150,  180,  210,  and  240.  The  pigeons  could  either 
complete  the  FR  schedule  presented  to  them,  or  they  could 
peck  the  CO  Key  and  switch  to  the  MR  schedule.  Once  a 
subject  began  responding  on  a  schedule,  the  CO  Key  was 
turned  off  and  the  subject  was  locked  into  that  schedule. 
When  only  one  response  on  the  CO  Key  was  required  to  switch 
schedules,  both  birds  switched  to  the  MR  alternative  at  an 
extremely  high  frequency;  they  remained  with  the  FR  schedule 
only  with  the  two  or  three  shortest  FR  values.  However,  in 
order  to  maximize  overall  rate  of  rei nforcement ,  subjects 
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should  have  switched  to  the  MR  schedule  only  when  the  FR 
value  was  greater  than  120.  Thus,  MR  was  preferred  over  FR 
more  than  would  be  predicted  by  relative  rates  of 
rei nforcement .  In  accordance  with  Findley  (1958),  Sherman 
and  Thomas  also  found  that  as  the  switching  requirement 
increased,  preference  for  the  MR  schedule  decreased. 

Preference  for  a  mu  1 1 i p 1 e- r at i o  versus  an  FR  schedule 
was  investigated  by  Boeving  and  Randolph  (1975).  The 
initial  links  consisted  of  FRIO  schedules.  The  terminal 
links  consisted  of  an  FR30  schedule,  and  a  multiple-ratio 
schedule  with  component  values  of  5  and  80.  Across 
conditions,  the  smaller  component  was  reduced  in  probability 
of  occurrence  from  .50  to  .00.  The  three  pigeons 
demonstrated  almost  exclusive  preference  for  the  multiple 
schedule,  except  when  the  probability  of  the  shorter 
component  was  reduced  to  .00.  At  that  point,  they  showed 
exclusive  preference  for  the  FR  alternative.  The  authors 
interpret  the  pigeons  preference  for  the  multiple  schedule 
as  an  instance  of  gambling  for  the  shortest  path  to 
reinforcement,  even  when  the  possibility  of  winning  the 
gamble  was  extremely  small. 

Rider  (1983b)  allowed  rats  to  choose  between 
equiprobable  MR  and  FR  schedules  of  food  rei nforcement . 
These  schedules  were  presented  either  as  concurrent 
schedules,  or  as  terminal  link  alternatives  in  a 
concurrent-chains  procedure  with  FR1  initial  links. 
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Preference  was  measured  as  proportion  of  responses  on  each 
schedule  in  the  concurrent  procedure,  and  as  proportion  of 
initial  link  choice  responses  in  the  concurrent -chai ns 
procedure.  Six  out  of  seven  rats  displayed  systematic 
schedule  preferences .  Five  rats  consistently  preferred  an 
FR35  schedule  over  an  MR50  schedule  (components  of  1/99)  in 
the  concurrent  procedure,  but  showed  a  reversed  preference 
in  the  concur  rent -chai ns  procedure.  The  seventh  rat  showed 
a  similar  trend  when  FR  values  were  systematically  varied 
between  25,  35,  50,  and  80.  The  FR  schedules  were 
consistently  preferred  in  the  concurrent  procedure,  with  the 
exception  of  the  FR80  alternative,  while  the  MR50 
alternative  was  generally  preferred  in  the  concurrent -chai ns 
procedure.  These  results  demonstrate  that  responding  on  a 
schedule  is  not  equivalent  to  reponding  for  a  schedule,  and 
appears  to  justify  the  use  of  the  concurrent -chai ns 
procedure  as  a  measure  of  schedule  preference. 

Weiner  (1966)  used  the  switching  procedure  to  examine 
preference  in  adult  humans  (N=4)  for  VR40  versus  FR40 
schedules  of  points  delivery.  After  rei nforcement  on  one 
schedule,  say  FR,  the  other  schedule,  VR,  would  be 
automatically  presented.  A  response  on  the  CO  key  was  thus 
necessary  to  reinstate  the  previous  schedule.  Weiner  found 
no  preference  for  either  VR  or  FR.  A  reason  for  this  may  be 
that  he  instructed  the  subjects  to  "get  the  highest  score 
possible".  Relative  to  this  strategy,  responding  on  the  CO 
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Key  would  waste  time  and  reduce  overall  rei nforcement ; 
therefore,  switching  would  not  occur.  This  assumes  that 
subjects  discriminated  the  VR  and  FR  schedules  as  equivalent 
in  mean  rate  of  rei nforcement .  In  other  conditions  of  this 
experiment,  subjects  did  show  preference  for  an  FRIO  over  an 
FR40  schedule,  and  for  an  FR40  schedule  which  resulted  in 
400  points  as  opposed  to  an  FR40  which  resulted  in  100 
points.  Both  of  these  preferences  are  consonant  with  the 
instruction  to  maximize  overall  points. 

Repp  and  Deitz  (1975)  used  a  switching  procedure  to 
investigate  human  preference  for  VR60  versus  FR60  schedules 
of  token  rei nforcement .  Each  token,  when  earned,  could  be 
immediately  exchanged  for  a  penny.  The  VR  and  FR  schedules 
alternated  automatically  after  each  rei nforcement ,  and  a 
switching  response  was  required  to  reinstate  the  previous 
schedule.  When  preferences  stabilized  at  a  certain  CO 
requirement,  the  requirement  was  increased  by  one  response. 
This  was  done  until  switching  behavior  was  eliminated.  The 
subjects,  two  boys  and  two  girls,  age  10  to  12,  all  switched 
to  the  VR  schedule  more  often  than  to  the  FR  schedule.  The 
relative  change-overs  to  VR  as  opposed  to  FR  were  always 
greater  than  .60  as  long  as  switching  behavior  was  being 
engaged  in.  All  subjects  abruptly  stopped  switching, 
however,  once  the  CO  requirement  requirement  reached  a 
certain  level:  9  responses  with  one  subject,  and  11,  13,  and 
17  responses  with  the  other  three.  This  result  conficts 
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with  previous  research  (e.g.,  Findley,  1958;  Sherman  & 
Thomas,  1968)  which  found  that  switching  would  gradually 
decrease  as  the  CO  requirement  increased.  Finally,  the 
question  arises  as  to  why  preference  for  VR  over  FR  was 
found  with  humans  in  this  study  and  not  in  the  similar  study 
by  Weiner  (1966).  One  factor  may  be  that  Repp  and  Deitz  did 
not  specifically  instruct  subjects  to  earn  as  much  as 
possible.  As  previously  discussed,  this  instruction  may 
disrupt  switching,  since  it  would  waste  time  and  reduce 
overall  earnings.  The  other  factor  may  be  the  relative  ages 
of  the  subjects;  Weiner  used  adults  while  Repp  and  Deitz 
used  children.  Research  has  shown  that  age  is  inversely 
related  to  preference  for  immediate  over  delayed 
rei nforcement  (e.g.,  Mischel  &  Metzner,  1962).  Thus, 
children  may  be  more  likely  than  adults  to  "weight"  the 
shorter  intervals,  i.e.,  the  more  immediate  rei nforcement , 
in  the  VR  schedule  more  heavily.  This  would  then  result  in 
preference  for  the  variable  alternative. 

Preference  for  Fixed  versus  Variable  Delay  of  Reinforcement 

Two  early  studies  (Logan,  1965;  Pubols,  1962)  examined 
preference  for  mixed  versus  fixed  delay  of  rei nforcement 
using  maze  running  procedures  with  rats.  Pubols  found  that 
rats  consistently  chose  that  arm  of  a  Y  maze  where 
reinforcment  was  delivered  after  a  mixed  delay  as  opposed  to 
the  other  arm  where  reinforcement  was  delivered  after  a 
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constant  delay.  The  component  values  of  the  mixed  delay 
were  zero  seconds  and  twice  the  value  of  the  constant  delay, 
and  were  presented  in  random  order .  The  rats  preferred  the 
mixed  delay  side  even  when  reinforcement  was  presented  only 
during  the  short  component,  i.e.,  immediate  rei nforcement  on 
50%  of  trials,  and  long  delay  but  no  rei nforcement  on  the 
other  50%  of  trials.  Logan  varied  both  amount  and  delay  of 
reward,  and  also  found  preference  for  mixed  over  fixed 
delay.  Six  out  of  seven  rats  preferred  a  mixed  delay 
alternative,  with  component  delays  of  0-  and  15-sec  (mean  = 
8-sec),  over  a  constant  delay  of  8-sec.  With  three  of  the 
rats,  preference  for  the  mixed  alternative  was  exclusive. 
The  results  also  indicated  that  the  mixed  delay  was 
approximately  equivalent  to  a  constant  delay  of  7-sec;  at 
that  value,  half  of  the  rats  preferred  the  mixed  alternative 
while  the  other  half  preferred  the  constant  alternative. 

Cicerone  (1976)  used  a  standard  concurrent-chains 
procedure  to  examine  preference  for  mixed  versus  constant 
delay  of  rei nforcement .  Pigeons  were  presented  various 
combinations  of  mixed  and  constant  delays,  and,  across 
conditions,  the  range  of  the  mixed  delay  values  was 
increased.  When  the  constant  delay  was  8-sec,  four  pigeons 
consistently  preferred  the  mixed  delay  alternative  when  it 
consisted  of  2-  and  14-sec  components,  but  not  when  it 
consisted  of  6-  and  10-sec  components.  Two  other  pigeons 
were  exposed  to  a  constant  delay  alternative  of  30-sec,  and 
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a  mixed  delay  alternative  with  either  15/45,  5/55,  or 
0/60-sec  components.  The  mixed  alternative  was  consistently 
preferred  only  for  the  two  broadest  ranges;  for  the  5/55  and 
0/60  mixed  delays,  the  respective  relative  rates  of 
responding  in  the  initial  links  were  .69  and  .97  for  one 
subject,  and  .93  and  .99  for  the  other  subject. 

Rider  (1983a)  examined  preference  for  mixed  versus 
constant  delay,  and  varied  the  probability  of  the  short 
component  of  the  mixed  delay.  Rats  were  run  on  a  standard 
concur rent -cha i ns  procedure,  with  the  addition  of  a  2-sec 
COD  in  the  initial  links.  The  constant  delay  was  15-sec  for 
three  subjects,  and  30-sec  for  two  subjects.  The  mixed 
delay  components  were  always  0.2-sec  and  twice  the  length  of 
the  constant  delay.  Across  conditions,  the  probability  of 
occurrence  of  the  short  component  was  .00,  .10,  .25,  .50, 
.75,  .90,  and  1.00.  In  all  cases,  relative  rate  of  initial 
link  responding  was  greater  for  the  mixed  alternative  when 
the  probability  of  the  short  component  was  above  .25.  In 
addition,  preference  for  the  mixed  alternative  was  much 
greater  for  the  two  rats  exposed  to  the  longer  delays.  A 
harmonic  transformat  ion  of  i nter rei nforcement  intervals, 
similar  to  Killeen  (1968),  was  attempted,  but  was  found  to 
be  inadequate. 
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Subs  t  an  t i ve  I ssues 

The  preceding  studies  have  generally  found  that 
variable  schedules  are  preferred  over  fixed  schedules  of 
rei nforcement .  With  respect  to  factors  determining  such 
preference,  a  good  deal  of  emphasis  has  been  placed  on 
Herrnstein' s  (1964b)  hypothesis  that  preference  is  a 
function  of  the  heavier  weighting  of  the  shorter  components 
in  the  variable  schedule.  A  number  of  transformation  rules 
have  been  attempted  to  precisely  describe  this  relationship, 
but,  as  Frankel  and  Vom  Saal  note,  "no  consistent 
transformation  has  been  found  which  is  adequate  to  account 
for  more  than  a  limited  set  of  data"  (Frankel  &  Vom  Saal, 
1976).  Rider  (1983a)  suggests  that  it  may  be  unreasonable 
to  expect  that  a  single  model  would  ever  be  entirely 
adequate.  He  supports  Navarick's  and  Fantino's  (1972) 
opinion  that  the  failure  of  schedule  preferences  to  show 
strong  stochastic  transitivity  implies  that  any  specific 
transformation  must  necessarily  be  limited  in  its 
applicability.  Of  relevance  to  this,  Herrnstein  (1981)  has 
recently  suggested  a  theoretical  model  regarding  preference 
for  large  delayed  versus  small  immediate  reinforcers.  Built 
into  the  model  is  the  assumption  that  the  effects  of  delayed 
reinforcement  on  response  strength  may  vary  as  a  function  of 
experiential  factors  and  genetic  differences  between  and 
within  species.  In  other  words,  the  subject's  "weighting" 
of  the  i nter rei nforcement  intervals  can  vary  as  a  result  of 
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a  number  of  factors,  and  cannot  be  expected  to  remain 
constant.  Thus,  Herrnstein' s  model  also  argues  against  the 
possibility  of  finding  a  precise,  yet  generally  applicable, 
transformat  ion  rule  for  describing  schedule  preferences. 

Variable  schedules  differ  from  fixed  schedules  not  only 
in  the  length  of  the  component  values,  but  also  in  the 
predictability  of  those  values.  Predictability  is  here 
defined  as  the  degree  to  which  the  components  are  correlated 
with  some  preceding  stimulus  or  event.  Fixed  schedules  are 
inherently  more  predictable  than  variable  schedules.  In 
addition,  multiple  schedules,  where  a  separate  cue  is 
provided  for  each  component  of  the  schedule,  are  more 
predictable  than  mixed  or  variable  schedules,  where  all 
components  are  presented  with  the  same  cue.  FranKel  and  Vom 
Saal  (1976),  as  well  as  Hendry  (1969),  found  some  evidence 
that  multiple  schedules  are  preferred  over  fixed  schedules 
more  so  than  mixed  schedules  are.  FranKel  and  Vom  Saal 
interpret  this  result  as  indicating  that  the 
nonpredictability  of  component  values  in  the  mixed  schedule 
only  serves  to  reduce  preference  for  that  schedule.  In 
other  words,  the  mixed  schedule  is  preferred  solely  on  the 
basis  of  the  heavier  weighting  of  the  shorter  component 
values,  and  the  nonpredictability  only  detracts  from  such 
preference.  A  problem  with  this  i nterpretat ion  is  that,  in 
their  concurrent  chains  procedure,  the  predictive  cues  of 
the  multiple  schedule  were  available  only  after  that 
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schedule  had  been  selected,  i.e.,  only  after  the  terminal 
link  was  entered.  Thus,  during  the  initial  link  choice 
phase,  the  upcoming  component  on  the  multiple  schedule  was 
still  as  nonpredictable  as  in  the  mixed  schedule.  Thus, 
even  in  selecting  the  multiple  schedule,  the  pigeons  were 
still  "gambling"  on  which  component  would  occur.  In  this 
manner,  preference  for  the  multiple  schedule  seems  to  be  an 
instance  of  "wanting  to  know  the  outcome  of  the  gamble"  as 
soon  as  possible,  rather  than  trying  to  avoid  the  gamble. 
Indeed,  Herrnstein  (1964b)  and  Boeving  and  Randolph  (1975) 
note  the  direct  similarity  of  preference  for  variable  over 
fixed  schedules  to  gambling  behavior.  In  addition,  Sherman 
and  Thomas  (1968)  demonstrated  that  pigeons  would  reliably 
switch  from  all  but  the  shortest  FR  components  of  a  multiple 
schedule  to  a  nonpredictable  mixed  schedule  even  when 
switching  reduced  the  rate  of  rei nforcement .  Their  results 
imply  that  preference  for  the  variable  schedule  is  a  direct 
function  of  gambling  for  the  shortest  path  to  rei nforcement . 

A  third  issue  concerns  the  generality  of  preference  for 
variable  over  fixed  schedules  across  species,  notably  to 
humans.  Only  two  studies  have  used  human  subjects  (Repp  & 
Deitz,  1975;  Weiner,  1966),  and  the  results  of  these  studies 
have  been  inconclusive.  Yet,  Emerson  (1972)  has 
hypothesized  that  preference  for  variable  over  fixed 
schedules  of  rei nforcement  may  be  an  important  factor  in 
human  social  interaction.  Individuals  who  deliver  social 
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reinforcement  on  variable  schedules  may  exert  greater 
influence  over  others  and  come  to  dominate  social 

relationships.  Thus,  research  on  schedule  preferences  in 
humans  may  lead  to  understanding  some  important  determinants 
of  human  behavior.  Relative  to  this  possibility,  however, 
the  efforts  to  conduct  such  research  have  been  inadequate. 
Conclusion 

Research  was  reviewed  concerning  preference  for 
variable  over  fixed  schedules  of  reinforcement.  Such 
preference  appears  to  be  a  consistent  finding  in  lower 
animals,  and  seems  to  be  a  function  of  a  tendency  to 
"gamble"  for  the  shorter  path  to  reinforcement  available 
with  the  variable  alternative.  It  was  noted,  however,  that 
research  with  humans  in  this  area  has  been  inadequate,  and 
the  studies  which  have  been  conducted  have  not  yielded 
consistent  results.  Interestingly,  Wearden  (Note  1) 
believes  that  much  of  the  animal  research  in  operant 
conditioning  may  have  little  relevance  to  human  behavior. 
Thus,  it  is  important  that  research  be  carried  out  to 
directly  determine  whether  schedule  preferences  found  in 
laboratory  animals  do  indeed  have  applicability  to  humans. 
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Reference  Notes 
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Appendix  B:  Questionnaires  Used  in  Experiment  3 


First  questionnaire: 


Subject  # _ 

This  study  is  concerned  with  individual  preferences. 
It  consists  of  filling  out  a  short  questionnaire,  for  which 
you  will  be  paid  $10.00.  Part  of  the  study,  however,  deals 
with  the  manner  in  which  you  will  be  paid  the  $10.00.  More 
specifically,  there  are  two  alternative  ways  in  which  this 
money  can  be  paid  to  you: 

A 1 ternat i ve  A:  You  will  receive  the  money  in  one 
mon  t  h ' s  time. 

A 1 ternat i ve  B:  You  will  receive  the  money  either  today 
or  in  two  month's  time.  This  will  be  determined  by  a 
coin  flip  such  that  you  have  a  50%  chance  of  receiving 
the  money  today  versus  in  two  months. 

(Please  note  that  whichever  alternative  you  select,  you  wi 1 1 
be  paid  the  money.  The  two  alternatives  differ  only  in 
terms  of  when  you  will  receive  the  money.) 

Please  indicate  how  you  wish  to  be  paid  by  placing  a 
checkmark  in  the  appropriate  space  below: 


Alternative  Time  of  Payment 


A 


In  one  month 


B  E i ther  today  or  i n 

2  months  ( to  be 
determined  by  coin 
flip) . 


Choice  (  X  ) 

(  ) 
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Second  questionnaire: 


Subject  # _ 

You  have  now  made  your  choice  between  the  two 
methods  of  payment,  and  your  answers  to  the  following  items 
will  have  no  further  bearing  on  that  choice. 

Nevertheless,  please  answer  the  following  items  as 
accurate  1 y 
as  possible. 

For  each  item,  please  rate  your  degree  of  preference  by 
circling  the  appropriate  number  on  the  rating  scale 
provided . 

For  example: 

Slightly  Strongly 

prefer  1  2  3  4  5  6  (7J  prefer 

(where  a  rating  of  7  indicates  strong  preference). 


1.  Would  you  prefer  to  receive  the  $10.00  today 
rather  than  in  one  month? 

Yes  _  No  _  If  you  answered  yes,  how 

strong  is  this  preference? 

Slightly  Strongly 

prefer  1  234567  prefer 

2.  Would  you  prefer  to  receive  the  $10.00  in 
one  month  rather  than  in  two  months? 

Yes _  No _  If  you  answered  yes,  how 

strong  is  this  preference? 

Slightly  Strongly 

prefer  1  234567  prefer 

3.  Would  you  prefer  to  receive  the  $10.00  today 
rather  than  in  two  months?  Yes No 

If  you  answered  yes,  how  strong  is  this 
preference? 

Slightly  Strongly 

prefer  1234567  prefer 


In  a  couple  of  sentences,  outline  your  reason 

for  having  chosen  the  method  of  payment  that  you  did: 
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